AI-powered web scraping CLI by ScrapeGraph AI. Get an API key at dashboard.scrapegraphai.com.
Works with
AI-first code editor with Composer
Before installing skills in Cursor, ensure your development environment meets these requirements:
node --versionjust-scrapeExecute the skills CLI command in your project's root directory to begin installation:
Fetches just-scrape from scrapegraphai/just-scrape and configures it for Cursor.
The CLI shows a list of agents. Use arrow keys and space to select Cursor:
Confirm successful installation by checking the skill directory location:
Restart Cursor to activate just-scrape. Access via /just-scrape in your agent's command palette.
We perform automated surface-level scans (Gen AI Scanner, Socket, Snyk) during installation. These checks detect common vulnerabilities but do not guarantee complete security. Always review skill source code and verify the publisher's reputation before production use.
Skills execute code in your environment. Always review source, verify the publisher, and test in isolation before production.
Submit your Claude Code skill and start earning
Create detailed user stories, acceptance criteria, and feature specs
Example
Generate user stories for 'password reset feature' with acceptance criteria, edge cases, and test scenarios
Reduce spec writing time by 50%, ensure comprehensive coverage
Research competitors, compare features, identify gaps
Example
Analyze 5 competitor products, create feature comparison matrix, suggest differentiation opportunities
Complete competitive research in 2 hours instead of 2 days
Evaluate features using frameworks (RICE, ICE, Kano) and create prioritized backlogs
Example
Score 20 feature ideas using RICE framework, generate prioritized roadmap with rationale
0
total installs
0
this week
11
GitHub stars
0
upvotes
Run in your terminal
0
installs
0
this week
11
stars
AI-powered web scraping CLI by ScrapeGraph AI. Get an API key at dashboard.scrapegraphai.com.
Always install or run the @latest version to ensure you have the most recent features and fixes.
npm install -g just-scrape@latest # npm
pnpm add -g just-scrape@latest # pnpm
yarn global add just-scrape@latest # yarn
bun add -g just-scrape@latest # bun
npx just-scrape@latest --help # run without installing
bunx just-scrape@latest --help # run without installing (bun)
export SGAI_API_KEY="sgai-..."
API key resolution order: SGAI_API_KEY env var → .env file → ~/.scrapegraphai/config.json → interactive prompt (saves to config).
| Need | Command |
|---|---|
| Extract structured data from a known URL | smart-scraper |
| Search the web and extract from results | search-scraper |
| Convert a page to clean markdown | markdownify |
| Crawl multiple pages from a site | crawl |
| Get raw HTML | scrape |
| Automate browser actions (login, click, fill) | agentic-scraper |
| Generate a JSON schema from description | generate-schema |
| Get all URLs from a sitemap | sitemap |
| Check credit balance | credits |
| Browse past requests | history |
| Validate API key | validate |
All commands support --json for machine-readable output (suppresses banner, spinners, prompts).
Scraping commands share these optional flags:
--stealth — bypass anti-bot detection (+4 credits)--headers <json> — custom HTTP headers as JSON string--schema <json> — enforce output JSON schemaExtract structured data from any URL using AI.
just-scrape smart-scraper <url> -p <prompt>
just-scrape smart-scraper <url> -p <prompt> --schema <json>
just-scrape smart-scraper <url> -p <prompt> --scrolls <n> # infinite scroll (0-100)
just-scrape smart-scraper <url> -p <prompt> --pages <n> # multi-page (1-100)
just-scrape smart-scraper <url> -p <prompt> --stealth # anti-bot (+4 credits)
just-scrape smart-scraper <url> -p <prompt> --cookies <json> --headers <json>
just-scrape smart-scraper <url> -p <prompt> --plain-text
# E-commerce extraction
just-scrape smart-scraper https://store.example.com/shoes -p "Extract all product names, prices, and ratings"
# Strict schema + scrolling
just-scrape smart-scraper https://news.example.com -p "Get headlines and dates" \
--schema '{"type":"object","properties":{"articles":{"type":"array","items":{"type":"object","properties":{"title":{"type":"string"},"date":{"type":"string"}}}}}}' \
--scrolls 5
# JS-heavy SPA behind anti-bot
just-scrape smart-scraper https://app.example.com/dashboard -p "Extract user stats" \
--stealth
Search the web and extract structured data from results.
just-scrape search-scraper <prompt>
just-scrape search-scraper <prompt> --num-results <n> # sources to scrape (3-20, default 3)
just-scrape search-scraper <prompt> --no-extraction # markdown only (2 credits vs 10)
just-scrape search-scraper <prompt> --schema <json>
just-scrape search-scraper <prompt> --stealth --headers <json>
# Research across sources
just-scrape search-scraper "Best Python web frameworks in 2025" --num-results 10
# Cheap markdown-only
just-scrape search-scraper "React vs Vue comparison" --no-extraction --num-results 5
# Structured output
just-scrape search-scraper "Top 5 cloud providers pricing" \
--schema '{"type":"object","properties":{"providers":{"type":"array","items":{"type":"object","properties":{"name":{"type":"string"},"free_tier":{"type":"string"}}}}}}'
Convert any webpage to clean markdown.
just-scrape markdownify <url>
just-scrape markdownify <url> --stealth # +4 credits
just-scrape markdownify <url> --headers <json>
just-scrape markdownify https://blog.example.com/my-article
just-scrape markdownify https://protected.example.com --stealth
just-scrape markdownify https://docs.example.com/api --json | jq -r '.result' > api-docs.md
Crawl multiple pages and extract data from each.
just-scrape crawl <url> -p <prompt>
just-scrape crawl <url> -p <prompt> --max-pages <n> # default 10
just-scrape crawl <url> -p <prompt> --depth <n> # default 1
just-scrape crawl <url> --no-extraction --max-pages <n> # markdown only (2 credits/page)
just-scrape crawl <url> -p <prompt> --schema <json>
just-scrape crawl <url> -p <prompt> --rules <json> # include_paths, same_domain
just-scrape crawl <url> -p <prompt> --no-sitemap
just-scrape crawl <url> -p <prompt> --stealth
# Crawl docs site
just-scrape crawl https://docs.example.com -p "Extract all code snippets" --max-pages 20 --depth 3
# Filter to blog pages only
just-scrape crawl https://example.com -p "Extract article titles" \
--rules '{"include_paths":["/blog/*"],"same_domain":true}' --max-pages 50
# Raw markdown, no AI extraction (cheaper)
just-scrape crawl https://example.com --no-extraction --max-pages 10
Get raw HTML content from a URL.
just-scrape scrape <url>
just-scrape scrape <url> --stealth # +4 credits
just-scrape scrape <url> --branding # extract logos/colors/fonts (+2 credits)
just-scrape scrape <url> --country-code <iso>
just-scrape scrape https://example.com
just-scrape scrape https://store.example.com --stealth --country-code DE
just-scrape scrape https://example.com --branding
Browser automation with AI — login, click, navigate, fill forms. Steps are comma-separated strings.
just-scrape agentic-scraper <url> -s <steps>
just-scrape agentic-scraper <url> -s <steps> --ai-extraction -p <prompt>
just-scrape agentic-scraper <url> -s <steps> --schema <json>
just-scrape agentic-scraper <url> -s <steps> --use-session # persist browser session
# Login + extract dashboard
just-scrape agentic-scraper https://app.example.com/login \
-s "Fill email with [email protected],Fill password with secret,Click Sign In" \
--ai-extraction -p "Extract all dashboard metrics"
# Multi-step form
just-scrape agentic-scraper https://example.com/wizard \
-s "Click Next,Select Premium plan,Fill name with John,Click Submit"
# Persistent session across runs
just-scrape agentic-scraper https://app.example.com \
-s "Click Settings" --use-session
Generate a JSON schema from a natural language description.
just-scrape generate-schema <prompt>
just-scrape generate-schema <prompt> --existing-schema <json>
just-scrape generate-schema "E-commerce product with name, price, ratings, and reviews array"
# Refine an existing schema
just-scrape generate-schema "Add an availability field" \
--existing-schema '{"type":"object","properties":{"name":{"type":"string"},"price":{"type":"number"}}}'
Get all URLs from a website's sitemap.
just-scrape sitemap <url>
just-scrape sitemap https://example.com --json | jq -r '.urls[]'
Browse request history. Interactive by default (arrow keys to navigate, select to view details).
just-scrape history <service> # interactive browser
just-scrape history<Make data-driven prioritization decisions faster
Draft PRDs, status updates, and stakeholder presentations
Example
Create executive summary of Q3 roadmap, monthly progress report, feature launch announcement
Save 3-5 hours/week on communication overhead
Prerequisites
Time Estimate
30-60 minutes to see productivity improvements
Steps
Common Pitfalls
✓ Do
✗ Don't
💡 Pro Tips
✓ Use when
Use for user story writing, competitive research, roadmap prioritization, stakeholder communication, and PRD drafting. Best for reducing repetitive documentation and research work.
✗ Avoid when
Avoid for strategic product vision (requires deep customer empathy), pricing decisions (needs market and financial expertise), or when face-to-face customer discovery is more valuable than speed.
mattpocock/skills
parcadei/continuous-claude-v3
cursor/plugins
ailabs-393/ai-labs-claude-skills
pproenca/dot-skills
ailabs-393/ai-labs-claude-skills
Useful defaults in just-scrape — fewer surprises than typical one-off scripts, and it plays nicely with `npx skills` flows.
I recommend just-scrape for anyone iterating fast on agent tooling; clear intent and a small, reviewable surface area.
just-scrape reduced setup friction for our internal harness; good balance of opinion and flexibility.
We added just-scrape from the explainx registry; install was straightforward and the SKILL.md answered most questions upfront.
Registry listing for just-scrape matched our evaluation — installs cleanly and behaves as described in the markdown.
just-scrape is among the better-maintained entries we tried; worth keeping pinned for repeat workflows.
Solid pick for teams standardizing on skills: just-scrape is focused, and the summary matches what you get after install.
Useful defaults in just-scrape — fewer surprises than typical one-off scripts, and it plays nicely with `npx skills` flows.
just-scrape is among the better-maintained entries we tried; worth keeping pinned for repeat workflows.
Registry listing for just-scrape matched our evaluation — installs cleanly and behaves as described in the markdown.
showing 1-10 of 74