ScrapeSome: Effortless Web Scraping for JavaScript-Heavy Sites

>
Tired of 403s and blank pages when scraping JavaScript-heavy websites?

You're not alone — and that's exactly why I built ScrapeSome.

🚀 What Is ScrapeSome?

ScrapeSome is a developer-friendly Python library that makes scraping modern websites simple — even the ones loaded with dynamic JavaScript or tough anti-bot protections.

It combines:

• ✅ Sync and async support
• 🎭 Automatic Playwright fallback for headless browser rendering
• 💻 CLI support: scrape straight from your terminal
• 🛡️ Built-in error handling, timeouts, and retries
• 📄 Output formats: HTML, Markdown, text, or JSON

It’s fast, lightweight, and requires zero boilerplate.

🔧 Why I Built It

I kept hitting walls on scraping projects:

• Pages rendered everything with JavaScript
• APIs were locked down or undocumented
• requests,Scrapy failed or got 403 request error
• Setting up full browser automation felt too heavy for small jobs

So I built ScrapeSome — to fill the gap between requests and full-on headless scraping frameworks.

⚙️ Quick Example

from scrapesome import sync_scraper

html = sync_scraper("https://example.com")

html = sync_scraper(
    "https://example.com",
    force_playwright=True,
    output_format="markdown",
    user_agents=["Mozilla/5.0"]
)

💻 CLI usage

scrapesome scrape --url https://example.com --output-format json

You can even configure behavior with environment variables — great for scripting.

📦 Install It

pip install scrapesome

🧪 Try it out on PyPI:

👉 https://pypi.org/project/scrapesome/

🔗 Links

• 🔧 GitHub: github.com/scrapesome/scrapesome
• 📚 Docs: scrapesome.onrender.com
• 📄 Full blog post: Medium

🙌 Feedback Welcome

This is an early release, and I’d love to hear your thoughts.

Try it, break it, file issues, suggest features — or just ⭐ the repo if you like the idea!

Happy scraping! 🕷️

— Vishnu Vardhan Reddy