Spider is a powerful web article parser that transforms cluttered web pages into clean, readable content.
Visit spider.jlopes.eu to try it out!
Spider supports multiple parsing strategies optimized for different types of websites:
- auto (default): Automatically selects the best strategy based on domain
- googlebot: Mimics Google's crawler for general content
- facebook: Uses Facebook's external hit agent
- archive: Optimized for archived or paywall content
You can extend Spider with custom parsers for specific domains by modifying the functions/node-fetch/node-fetch.mjs file.
- Postlight for the Mercury Parser
- Astro team for the amazing framework
- Tailwind CSS and DaisyUI for the styling foundation
- Netlify for seamless deployment