Reliable web scraping with cascading fallbacks, anti-bot bypass, and poison pill detection.
Automate repetitive workflows and reduce manual effort
Example: Generate reports, summarize documents, draft communications
Save 3-5 hours per week on routine tasks
Learn new skills, understand complex topics, get expert guidance
Example: Explain concepts, provide examples, suggest learning resources
Accelerate learning and skill development by 2x
Enhance output quality through reviews, suggestions, and refinements
Example: Review drafts, suggest improvements, catch errors
Improve work quality by 30-40% with less effort
Time estimate: 15-45 minutes depending on use case complexity
Prerequisites
Submit your Claude Code skill and start earning
epx add jamditis/claude-skills-journalism --skill web-scrapingCompatible with
Supported agents
Implements a scraping cascade architecture with four strategies: trafilatura for fast article extraction, requests with rotating user agents, Playwright with stealth mode for JavaScript-heavy sites, and async Playwright for Jupyter notebooks
Includes poison pill detection to identify paywalls, CAPTCHAs, rate limits, Cloudflare blocks, and login walls using pattern matching and status code analysi
AI-first code editor with Composer
Before installing skills in Cursor, ensure your development environment meets these requirements:
node --versionweb-scrapingExecute the skills CLI command in your project's root directory to begin installation:
epx add jamditis/claude-skills-journalism --skill web-scrapingFetches web-scraping from jamditis/claude-skills-journalism and configures it for Cursor.
The CLI shows a list of agents. Use arrow keys and space to select Cursor:
Confirm successful installation by checking the skill directory location:
.cursor/skills/web-scraping
Restart Cursor to activate web-scraping. Access via /web-scrapingin your agent's command palette.
Learn more:View source on GitHubSkills CLI docsAbout CursorWhat are agent skills?
Steps
Common Pitfalls
✓ Do
✗ Don't
💡 Pro Tips
✓ Use when
Use when skill capabilities match your task, clear ROI on time saved, and you can validate outputs. Best for repetitive tasks, learning, and quality improvement.
✗ Avoid when
Avoid when task requires deep expertise you can't validate, involves sensitive decisions, or when learning process is more valuable than speed of completion.
apple-design
emilkowalski/skills · design · tag: web
develop-web-game
openai/skills · Productivity · tag: web
typescript-best-practices
jwynia/agent-skills · Backend · same category
python-backend-architecture-review
rknall/claude-skills · Backend · same category
django-expert
jeffallan/claude-skills · Backend · same category
google-flights
skillhq/flight-search · Backend · same category
Useful defaults in web-scraping — fewer surprises than typical one-off scripts, and it plays nicely with `npx skills` flows.
web-scraping has been reliable in day-to-day use. Documentation quality is above average for community skills.
Keeps context tight: web-scraping is the kind of skill you can hand to a new teammate without a long onboarding doc.
web-scraping is among the better-maintained entries we tried; worth keeping pinned for repeat workflows.
We added web-scraping from the explainx registry; install was straightforward and the SKILL.md answered most questions upfront.
web-scraping reduced setup friction for our internal harness; good balance of opinion and flexibility.
Useful defaults in web-scraping — fewer surprises than typical one-off scripts, and it plays nicely with `npx skills` flows.
We added web-scraping from the explainx registry; install was straightforward and the SKILL.md answered most questions upfront.
web-scraping reduced setup friction for our internal harness; good balance of opinion and flexibility.
web-scraping is among the better-maintained entries we tried; worth keeping pinned for repeat workflows.
showing 1-10 of 48