browser-automationsearch-web

Bright Data

brightdata

by brightdata

Access real-time web scraping with Bright Data. Scrape any website and extract structured data easily using advanced web

Integrates with Bright Data's web scraping infrastructure to provide real-time access to public web data through specialized tools for search engine scraping, webpage extraction, and structured data retrieval from popular websites.

github stars

2.2K

0 commentsdiscussion

Both formats append explainx.ai attribution and the canonical URL for this MCP server listing.

5,000 requests/month free tierAnti-blocking infrastructureReal-time data access

best for

  • / AI applications needing live web data
  • / Market research and competitive analysis
  • / Content aggregation and monitoring
  • / Building AI agents with web access

capabilities

  • / Scrape Google search results
  • / Extract data from any webpage
  • / Retrieve structured data from popular websites
  • / Access real-time web content
  • / Bypass anti-scraping protections
  • / Search engine results extraction

what it does

Provides real-time web scraping capabilities through Bright Data's infrastructure, allowing AI to extract data from websites without getting blocked.

about

Bright Data is an official MCP server published by brightdata that provides AI assistants with tools and capabilities via the Model Context Protocol. Access real-time web scraping with Bright Data. Scrape any website and extract structured data easily using advanced web It is categorized under browser automation, search web.

how to install

You can install Bright Data in your AI client of choice. Use the install panel on this page to get one-click setup for Cursor, Claude Desktop, VS Code, and other MCP-compatible clients. This server runs locally on your machine via the stdio transport.

license

MIT

Bright Data is released under the MIT license. This is a permissive open-source license, meaning you can freely use, modify, and distribute the software.

readme

Bright Data Logo

The Web MCP

🌐 Give your AI real-time web superpowers
Seamlessly connect LLMs to the live web without getting blocked

npm version npm downloads License

Quick StartFeaturesPricingDemosDocsSupport

🎉 Free Tier Available! 🎉

5,000 requests/month FREE
Perfect for prototyping and everyday AI workflows


--- ## 🌟 Overview **The Web MCP** is your gateway to giving AI assistants true web capabilities. No more outdated responses, no more "I can't access real-time information" - just seamless, reliable web access that actually works. Built by [Bright Data](https://brightdata.com), the world's #1 web data platform, this MCP server ensures your AI never gets blocked, rate-limited, or served CAPTCHAs.
Works with Any LLM
Claude, GPT, Gemini, Llama
🛡️ Never Gets Blocked
Enterprise-grade unblocking
🚀 5,000 Free Requests
Monthly
Zero Config
Works out of the box
--- ## 🎯 Perfect For - 🔍 **Real-time Research** - Get current prices, news, and live data - 🛍️ **E-commerce Intelligence** - Monitor products, prices, and availability - 📊 **Market Analysis** - Track competitors and industry trends - 🤖 **AI Agents** - Build agents that can actually browse the web - 📝 **Content Creation** - Access up-to-date information for writing - 🎓 **Academic Research** - Gather data from multiple sources efficiently --- ## ⚡ Quick Start **Use the configuration wizard:** ![GIF for day2](https://github.com/user-attachments/assets/b3917553-6cf9-4264-bc7a-9b8b74df0a17) 📡 Use our hosted server - No installation needed! Perfect for users who want zero setup. Just add this URL to your MCP client: ``` https://mcp.brightdata.com/mcp?token=YOUR_API_TOKEN_HERE ``` **Setup in Claude Desktop:** 1. Go to: Settings → Connectors → Add custom connector 2. Name: `Bright Data Web` 3. URL: `https://mcp.brightdata.com/mcp?token=YOUR_API_TOKEN` 4. Click "Add" and you're done! ✨ Run locally on your machine ```json { "mcpServers": { "Bright Data": { "command": "npx", "args": ["@brightdata/mcp"], "env": { "API_TOKEN": "" } } } } ``` --- ## 🚀 Pricing & Modes
⚡ Rapid Mode (Free tier) 💎 Pro Mode 🔧 Custom Mode

$0/month

5,000 requests


✅ Web Search
✅ Scraping with Web unlocker
❌ Browser Automation
❌ Web data tools


Default Mode

Pay-as-you-go

Everything in rapid plus 60+ tools


✅ Browser Control
✅ Web Data APIs



PRO_MODE=true

Usage-based

Pick the tools you need


✅ Combine tool groups
✅ Add individual tools
❌ Overrides Pro eligibility


GROUPS="browser"
TOOLS="scrape_as_html"
> **💡 Note:** Pro mode is **not included** in the free tier and incurs > additional charges based on usage. --- ## 🧠 Advanced Tool Selection - `GROUPS` lets you enable curated tool bundles. Use comma-separated group IDs such as `ecommerce,browser`. - `TOOLS` adds explicit tool names on top of the selected groups. - Mode priority: `PRO_MODE=true` (all tools) → `GROUPS` / `TOOLS` (whitelist) → default rapid mode (base toolkit). - Base tools always enabled: `search_engine`, `search_engine_batch`, `scrape_as_markdown`, `scrape_batch`. - Group ID `custom` is reserved; use `TOOLS` for bespoke picks.
Group ID Description Featured tools
ecommerce Retail and marketplace datasets web_data_amazon_product, web_data_walmart_product, web_data_google_shopping
social Social, community, and creator insights web_data_linkedin_posts, web_data_tiktok_posts, web_data_youtube_videos
browser Bright Data Scraping Browser automation tools scraping_browser_snapshot, scraping_browser_click_ref, scraping_browser_screenshot
finance Financial intelligence datasets web_data_yahoo_finance_business
business Company and location intelligence datasets web_data_crunchbase_company, web_data_zoominfo_company_profile, web_data_zillow_properties_listing
research News and developer data feeds web_data_github_repository_file, web_data_reuter_news
app_stores App store data web_data_google_play_store, web_data_apple_app_store
travel Travel information web_data_booking_hotel_listings
advanced_scraping Batch and AI-assisted extraction helpers search_engine_batch, scrape_batch, extract
### Claude Desktop example ```json { "mcpServers": { "Bright Data": { "command": "npx", "args": ["@brightdata/mcp"], "env": { "API_TOKEN": "", "GROUPS": "browser,advanced_scraping", "TOOLS": "extract" } } } } ``` --- ## ✨ Features ### 🔥 Core Capabilities
🔍 Smart Web Search
Google-quality results optimized for AI
📄 Clean Markdown
AI-ready content extraction
🌍 Global Access
Bypass geo-restrictions automatically
🛡️ Anti-Bot Protection
Never get blocked or rate-limited
🤖 Browser Automation
Control real browsers remotely (Pro)
Lightning Fast
Optimized for minimal latency
### 🎯 Example Queries That Just Work ```yaml ✅ "What's Tesla's current stock price?" ✅ "Find the best-rated restaurants in Tokyo right now" ✅ "Get today's weather forecast for New York" ✅ "What movies are releasing this week?" ✅ "What are the trending topics on Twitter today?" ``` --- ## 🎬 Demos > **Note:** These videos show earlier versions. New demos coming soon! 🎥
View Demo Videos ### Basic Web Search Demo https://github.com/user-attachments/assets/59f6ebba-801a-49ab-8278-1b2120912e33 ### Advanced Scraping Demo https://github.com/user-attachments/assets/61ab0bee-fdfa-4d50-b0de-5fab96b4b91d [📺 More tutorials on YouTube →](https://github.com/brightdata-com/brightdata-mcp/blob/main/examples/README.md)
--- ## 🔧 Available Tools ### ⚡ Rapid Mode Tools (Default - Free) | Tool | Description | Use ---

FAQ

What is the Bright Data MCP server?
Bright Data is a Model Context Protocol (MCP) server profile on explainx.ai. MCP lets AI hosts (e.g. Claude Desktop, Cursor) call tools and resources through a standard interface; this page summarizes categories, install hints, and community ratings.
How do MCP servers relate to agent skills?
Skills are reusable instruction packages (often SKILL.md); MCP servers expose live capabilities. Teams frequently combine both—skills for workflows, MCP for APIs and data. See explainx.ai/skills and explainx.ai/mcp-servers for parallel directories.
How are reviews shown for Bright Data?
This profile displays 56 aggregated ratings (sample rows for discoverability plus signed-in user reviews). Average score is about 4.5 out of 5—verify behavior in your own environment before production use.

Use Cases

Web Research & Information Gathering

Fetch and extract information from websites automatically

Example

Research competitor pricing, scrape product reviews, monitor news mentions

Automate 5-10 hours/week of manual web research

Content Monitoring & Alerts

Track website changes, new content, price updates

Example

Monitor competitor blog for new posts, track stock availability, watch for pricing changes

Stay informed without manual checking, never miss important updates

Data Extraction & Aggregation

Extract structured data from multiple websites

Example

Compile product listings from 10 e-commerce sites, aggregate job postings, collect real estate data

Build datasets 100x faster than manual copying

API-less Integration

Interact with services that don't offer APIs

Example

Check form submissions, validate website functionality, test user flows

Automate interactions with any website, even without API

Implementation Guide

Prerequisites

  • Claude Desktop or Cursor with MCP support
  • Understanding of web scraping ethics and robots.txt
  • Rate limiting awareness to avoid overwhelming target sites
  • Knowledge of legal restrictions on data collection

Time Estimate

20-40 minutes including configuration and testing

Installation Steps

  1. 1.Install web automation MCP server via npm or pip
  2. 2.Configure allowed domains and rate limits in MCP config
  3. 3.Test with simple fetch: 'Get content from example.com'
  4. 4.Progress to extraction: 'Extract all product prices from this page'
  5. 5.Set up monitoring: 'Check this URL daily for changes'
  6. 6.Parse structured data: 'Create CSV from this table'
  7. 7.Respect robots.txt and rate limits always

Troubleshooting

  • 403 Forbidden: Website blocks bots—respect their wishes, use official API instead
  • Rate limit errors: Slow down requests, add delays between fetches
  • Stale data: Target site changed HTML structure—update selectors
  • Timeout errors: Site is slow or blocking—increase timeout, try different user agent
  • JavaScript-rendered content: Use headless browser MCP servers for dynamic sites

Best Practices

✓ Do

  • +Check robots.txt and respect crawl rules
  • +Rate limit requests: 1-2 requests/second maximum
  • +Use official APIs when available instead of scraping
  • +Identify your bot with descriptive user agent
  • +Cache results to minimize repeated requests
  • +Handle errors gracefully with retries and fallbacks
  • +Validate extracted data for accuracy

✗ Don't

  • Don't scrape sites that explicitly forbid it (robots.txt, ToS)
  • Don't overwhelm servers with rapid requests—use rate limiting
  • Don't scrape personal data without consent and legal basis
  • Don't ignore copyright on extracted content
  • Don't assume HTML structure is stable—handle changes
  • Don't use scraped data for commercial purposes without permission

💡 Pro Tips

  • Use CSS selectors or XPath for robust data extraction
  • Set up monitoring alerts for extraction failures (structure changed)
  • Implement exponential backoff for retries on failures
  • Store raw HTML for reprocessing if extraction logic changes
  • Combine with data analysis tools for insights from extracted data
  • Consider using official APIs or RSS feeds as more stable alternatives

Technical Details

Architecture

MCP server handles HTTP requests, HTML parsing, JavaScript rendering (if headless browser), and returns structured data to Claude.

Protocols

  • HTTP/HTTPS
  • WebSocket (for real-time sites)
  • Puppeteer/Playwright (for JavaScript sites)

Compatibility

  • Static HTML sites
  • JavaScript-rendered SPAs (with headless browser)
  • REST APIs
  • GraphQL endpoints

When to Use This

✓ Use When

Use for research automation, content monitoring, data aggregation from multiple sources, and when official APIs don't exist. Best for read-only information gathering.

✗ Avoid When

Avoid for sites with APIs (use API instead), sites that explicitly forbid scraping, when data is copyrighted, or for login-required content without proper authorization.

Integration

  • Scheduled monitoring with change detection
  • Multi-source data aggregation pipelines
  • Fallback to web scraping when API rate limits hit
  • Headless browser for JavaScript-heavy sites

Discussion

Product Hunt–style comments (not star reviews)
  • No comments yet — start the thread.

List & Promote Your MCP Server

Share your MCP server with the developer community

GET_STARTED →
MCP server reviews

Ratings

4.556 reviews
  • Anika Thompson· Dec 20, 2024

    Bright Data is a well-scoped MCP server in the explainx.ai directory — install snippets and categories matched our Claude Code setup.

  • Mei Li· Dec 16, 2024

    Bright Data is among the better-indexed MCP projects we tried; the explainx.ai summary tracks the official description.

  • Harper Ghosh· Dec 16, 2024

    We wired Bright Data into a staging workspace; the listing’s GitHub and npm pointers saved time versus hunting across READMEs.

  • Pratham Ware· Dec 4, 2024

    According to our notes, Bright Data benefits from clear Model Context Protocol framing — fewer ambiguous “AI plugin” claims.

  • Anika Brown· Nov 11, 2024

    Bright Data is among the better-indexed MCP projects we tried; the explainx.ai summary tracks the official description.

  • Kofi Wang· Nov 7, 2024

    Bright Data is a well-scoped MCP server in the explainx.ai directory — install snippets and categories matched our Claude Code setup.

  • Neel Liu· Nov 7, 2024

    We evaluated Bright Data against two servers with overlapping tools; this profile had the clearer scope statement.

  • Nia Desai· Oct 26, 2024

    We wired Bright Data into a staging workspace; the listing’s GitHub and npm pointers saved time versus hunting across READMEs.

  • Neel Farah· Oct 26, 2024

    Bright Data is among the better-indexed MCP projects we tried; the explainx.ai summary tracks the official description.

  • Luis Chen· Oct 2, 2024

    We evaluated Bright Data against two servers with overlapping tools; this profile had the clearer scope statement.

showing 1-10 of 56

1 / 6