A modern web application for previewing links with real-time processing. Users can submit URLs, and Link Lens asynchronously extracts Open Graph metadata to display rich previews with images and titles. Built with a focus on performance, reliability, and user experience.
link-lens-video.mp4
- Getting Started
- Project Structure
- Key Features & Approach
- Tech Stack & Rationale
- AI Partnership
- Challenges & Solutions
- Testing & Quality
- Future Implementation
- Contact & Acknowledgements
link-lens-setup-video.mp4
- Node.js (v18+)
- npm or yarn
- Docker and Docker Compose
- Redis GUI (RedisInsight, Redis Desktop Manager) - to monitor job queues and worker processes
- PostgreSQL GUI (pgAdmin, TablePlus) - to inspect database records
What Redis does: Manages the job queue for asynchronous URL processing What PostgreSQL does: Persistent storage for URL submissions and their metadata
-
Clone and navigate to backend
git clone https://github.com/V3RS/link-lens.git cd link-lens cd backend
-
Environment setup
cp .env.example .env # Edit .env with your database credentials if needed -
Start services
docker-compose up -d # Note: PostgreSQL runs on port 5433 to avoid conflicts -
Install dependencies and setup database
npm install npm run prisma:migrate npm run prisma:generate
-
Start API server
npm run dev # API runs on http://localhost:3000 -
Start worker (in a second terminal)
cd backend npm run worker
-
Navigate to frontend
cd frontend -
Install and start
npm install npm run dev # Frontend runs on http://localhost:5173
3 terminals running (API server, worker, frontend dev server) and a working app at localhost:5173
- Open http://localhost:5173
- Enter a URL in the form (try: https://github.com, https://vsingh.dev)
- Backend queues and processes the URL asynchronously
- Frontend polls the API and shows real-time status updates
- View the extracted preview image, title, and submission history
link-lens/
├── backend/ # Node.js TypeScript API server
│ ├── src/
│ │ ├── app.ts # Express server setup & CORS config
│ │ ├── env.ts # Environment variable validation
│ │ ├── lib/
│ │ │ ├── db.ts # Prisma database client
│ │ │ ├── og.ts # Open Graph HTML parser
│ │ │ └── queue.ts # BullMQ queue configuration
│ │ └── routes/
│ │ └── submissions.ts # API endpoints with pagination & validation
│ ├── worker/
│ │ └── index.ts # Background job processor
│ ├── tests/ # Comprehensive test suite
│ │ ├── api.submissions.spec.ts # API endpoint tests
│ │ ├── db.integration.spec.ts # Database integration tests
│ │ ├── happy.e2e.spec.ts # End-to-end workflow tests
│ │ ├── parser.spec.ts # HTML parsing unit tests
│ │ ├── validation.spec.ts # Input validation tests
│ │ └── worker.spec.ts # Background worker tests
│ ├── prisma/
│ │ ├── schema.prisma # Database schema & relationships
│ │ └── migrations/ # Database version control
│ ├── docker-compose.yml # PostgreSQL & Redis services
│ └── package.json # Dependencies & scripts
│
└── frontend/ # React TypeScript SPA
├── src/
│ ├── App.tsx # Main application component
│ ├── components/
│ │ ├── Header.tsx # App header with theme toggle
│ │ ├── Footer.tsx # App footer with links
│ │ ├── SubmitForm.tsx # URL submission form
│ │ ├── LatestSubmission.tsx # Latest submission display
│ │ ├── SubmissionHistory.tsx # Paginated history view
│ │ ├── SubmissionCard.tsx # Individual submission display
│ │ ├── StatusBadge.tsx # Status indicator component
│ │ └── ThemeToggle.tsx # Dark/light theme switcher
│ ├── hooks/
│ │ ├── useSubmissions.ts # API integration & state management
│ │ ├── useTheme.ts # Theme persistence & switching
│ │ └── useLocalStorage.ts # LocalStorage utility hook
│ ├── constants/
│ │ └── index.ts # App-wide constants & configuration
│ ├── utils/
│ │ ├── url.ts # URL validation & formatting
│ │ └── date.ts # Date formatting utilities
│ ├── types.ts # TypeScript interface definitions
│ └── index.tsx # React app entry point
└── package.json # Dependencies & build scripts
- URL Submission - Simple form interface for entering links
- Async Processing - Background job processing with BullMQ
- Real-time Status Updates - Live polling shows QUEUED → PROCESSING → COMPLETE states
- Rich Previews - Extracts Open Graph images and titles
- Persistent History - All submissions saved with server-side pagination
- Error Handling - Graceful handling of failed requests and invalid URLs
- Theme Support - Dark/light mode toggle with CSS variables
I chose an asynchronous queue-based architecture over inline processing for several key reasons:
Fast Response Times: The API responds immediately when a URL is submitted, regardless of how long the external site takes to respond. This keeps the user interface snappy and responsive.
Reliability & Resilience: If an external site is slow or temporarily unavailable, it doesn't block other operations. The worker can retry failed jobs and handle errors gracefully without affecting the main application flow.
Scalability: The queue system allows for horizontal scaling - multiple workers can process jobs concurrently, and the system can handle traffic spikes by queueing requests.
Clear Status Flow: Users see exactly what's happening with their submission through clear status transitions, making the system transparent and debuggable.
- User submits URL → API validates and saves submission with
QUEUEDstatus - BullMQ worker picks up job → Changes status to
PROCESSING - Worker fetches HTML → Uses undici with timeouts and retries
- Extract Open Graph data → Parses HTML for
og:image,og:title, etc. - Update database → Status becomes
COMPLETE,NO_OG, orFAILED - Frontend polls API → Real-time updates shown to user with toast notifications
| Technology | Why I Picked It |
|---|---|
| Node.js + TypeScript | Type safety and modern JavaScript features for robust backend development |
| Express | Lightweight, flexible web framework with excellent middleware ecosystem |
| Prisma + PostgreSQL | Type-safe ORM with excellent developer experience and reliable relational database |
| BullMQ + Redis | Robust job queue system for handling asynchronous URL processing |
| undici | Modern, high-performance HTTP client with built-in timeout and retry capabilities |
| node-html-parser | Fast, lightweight HTML parser for extracting Open Graph metadata |
| Zod | Runtime type validation with excellent TypeScript integration |
| Vitest + Supertest | Modern testing framework with comprehensive API testing capabilities |
| React + Vite | Fast development experience with modern React tooling |
| Tailwind CSS | Utility-first CSS for rapid, consistent UI development |
I chose this stack based on both familiarity and practicality so I could give the project my best effort within the timebox. I wanted to strike a balance between moving quickly with tools I know well and making thoughtful, modern choices that align with production-ready practices.
BullMQ vs. Simple Job Processing: Chose BullMQ over simpler alternatives (like setTimeout or direct processing) for production-ready features like job persistence, retries, and monitoring capabilities.
Prisma vs. Raw SQL: Traded some performance for developer experience and type safety. Prisma's migrations and schema management significantly outweighed the slight query overhead.
Server-Side vs. Client-Side Pagination: Initially implemented client-side pagination for simplicity, but refactored to server-side for scalability - a good example of evolving architecture as requirements become clearer.
Polling vs. WebSockets: Chose intelligent polling over WebSockets to avoid connection management complexity while still providing real-time feel for the 95% use case where jobs complete quickly.
I partnered with AI tools throughout this project to accelerate development while maintaining high code quality. Cursor helped with architecture scaffolding and code generation, while I used various AI tools for research and problem-solving. The AI partnership enabled me to focus on architecture decisions and business logic rather than boilerplate, while ensuring I understood and could maintain every line of code.
Challenge: The initial client-side pagination was fetching all data and slicing it locally, which wouldn't scale and caused issues with the "latest submission" display.
Solution: Implemented proper offset-based server-side pagination. The API now accepts offset and limit parameters, and the frontend makes separate optimized calls: one for the latest submission (always item #1) and another for paginated history items. This reduced network calls by 50% during navigation and provides true scalability.
Challenge: Users needed to see status changes in real-time, but I wanted to avoid the complexity of WebSocket connections.
Solution: Implemented intelligent polling that only activates when there are active jobs (QUEUED or PROCESSING status). The frontend polls every 2 seconds during active processing and stops when all jobs complete. Combined with toast notifications, this provides a real-time feel with simple HTTP requests.
Challenge: External websites can be slow, return malformed HTML, or use relative URLs for images, leading to failed previews or broken image links.
Solution: Used undici with 7-second timeouts and proper error boundaries. Added URL validation to reject unsafe schemes like file://. For relative image URLs, implemented proper resolution using new URL(relative, base) to always generate absolute URLs.
I built this project using Test-Driven Development (TDD), writing tests before implementation. This approach proved invaluable for several reasons:
Clear Requirements: Writing tests first forced me to think through the exact behavior each function should have, leading to cleaner, more focused implementations.
Confidence in Refactoring: When implementing server-side pagination and optimizing network calls, the comprehensive test suite ensured no regressions were introduced during major architectural changes.
Better Error Handling: TDD naturally led to testing edge cases (invalid URLs, network failures, malformed HTML), resulting in more robust error handling than I would have implemented otherwise.
- Unit Tests: Core logic (HTML parsing, URL validation, worker processing)
- Integration Tests: Database operations with real PostgreSQL connections
- API Tests: Full HTTP request/response cycles with Supertest
- E2E Tests: Complete workflow from URL submission to final status updates
- Type Safety: Comprehensive TypeScript coverage with strict mode enabled
The test suite includes realistic scenarios like parsing HTML with missing Open Graph tags, handling network timeouts, and validating pagination edge cases. This comprehensive coverage caught several subtle bugs during development and gives confidence when deploying changes.
Run tests:
cd backend
npm test # Run all tests
npm run test:watch # Watch mode for development- Design Consistency - Refactor latest submission to use the same reusable
SubmissionCardcomponent as history items for UI consistency and smaller footprint - Favicon Integration - Extract and display website favicons alongside Open Graph data, persisted in database for visual richness
- Advanced Filtering - Add search functionality and status filters (complete, failed, etc.) with real-time results
- Bulk Operations - Multi-select submissions for batch delete, export, or re-processing actions
- Smart URL Deduplication - Check for existing URLs before creating new submissions, but maintain separate history entries for user context
- Soft Delete System - Add
deleted_attimestamp for submission removal without losing data, with toggle to show/hide deleted items - Redis Caching Layer - Cache frequently accessed submissions and Open Graph data to reduce database queries
- Image CDN Integration - Optimize and serve Open Graph images through a CDN with automatic resizing and WebP conversion
- WebSocket Real-time Updates - Replace polling with WebSocket connections for instant status updates and better scalability
- Rate Limiting - Implement per-IP submission limits and exponential backoff to prevent abuse
- API Versioning - Add
/v1/prefixed routes with backward compatibility for future API evolution - Analytics Dashboard - Track submission success rates, popular domains, processing times, and system health metrics
- User Authentication - Add user accounts with personal submission history and custom API rate limits
- Export Functionality - Generate CSV/JSON exports of submission history with filtering options
Don't hesitate to reach out if you want to chat about the project or anything engineering. I'm always excited to discuss technical decisions, architecture trade-offs, or potential improvements.