chore: update package-lock.json and remove stale requirements file by Muhammad-Hashim · Pull Request #3 · Muhammad-Hashim/llmstxt

Muhammad-Hashim · 2026-05-09T07:53:08Z

Description of change

Pull-Request Checklist

Code is up-to-date with the main branch
npm run lint passes with this change
npm run test passes with this change
This pull request links relevant issues as Fixes #0000
There are new or updated unit tests validating the change
Documentation has been updated to reflect this change
The new commits follow conventions outlined in the conventional commit spec

Summary by CodeRabbit

New Features
- Complete Next.js example application with multiple pages (home, about, features, pricing, blog, docs)
- Enhanced markdown middleware with improved link resolution support
- Front-matter metadata in generated llms-full.txt output
- Automatic localhost fallback for development environments
Documentation
- Expanded READMEs with setup guides and configuration examples for all packages
- New example application documentation with getting-started instructions
Improvements
- Better URL handling in markdown conversion for link processing
- Improved base URL environment variable resolution

- Add url parameter to htmlToMarkdown callback signature - Add extractHtmlTitle helper to pull page title from HTML - Emit title/url/description/page-title metadata per page in llms-full.txt - Fix htmlToMarkdown call sites to forward the page URL

…converter Add resolveHref() helper that converts relative hrefs to absolute URLs using the page URL as base. Update builtinHtmlToMarkdown to accept and forward pageUrl so links in Markdown output are always fully qualified.

…lhost fallback - Replace NextResponse with native Response in route handlers to avoid importing next/server unnecessarily - Add localhost:3000 fallback in resolveBaseUrlFromEnv when NODE_ENV=development

…ages - Add prepare script (npm run build) to core, middleware, and next packages - Add documentation, next-middleware, content-signal, route-handler keywords - Add @types/node devDependency to @llmstxt/next

- Document url parameter in htmlToMarkdown callback - Add Content-Signal header documentation to middleware README - Update code examples to match new API signatures - Add YAML front matter response example to middleware README

Includes a working Next.js application that demonstrates: - llms.txt and llms-full.txt route handlers via @llmstxt/next - Markdown content-negotiation via @llmstxt/middleware - Multiple page types: docs, blog, features, pricing, about, api-reference

coderabbitai · 2026-05-09T07:53:20Z

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Free

Run ID: a3e77e6d-6861-4026-8e80-4d23d26407eb

📥 Commits

Reviewing files that changed from the base of the PR and between 1c78751 and 0581745.

⛔ Files ignored due to path filters (8)

examples/nextjs-test/app/favicon.ico is excluded by !**/*.ico
examples/nextjs-test/package-lock.json is excluded by !**/package-lock.json
examples/nextjs-test/public/file.svg is excluded by !**/*.svg
examples/nextjs-test/public/globe.svg is excluded by !**/*.svg
examples/nextjs-test/public/next.svg is excluded by !**/*.svg
examples/nextjs-test/public/vercel.svg is excluded by !**/*.svg
examples/nextjs-test/public/window.svg is excluded by !**/*.svg
package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (37)

README.md
examples/README.md
examples/nextjs-test/.gitignore
examples/nextjs-test/AGENTS.md
examples/nextjs-test/CLAUDE.md
examples/nextjs-test/README.md
examples/nextjs-test/app/about/page.tsx
examples/nextjs-test/app/api-reference/page.tsx
examples/nextjs-test/app/blog/page.tsx
examples/nextjs-test/app/docs/getting-started/page.tsx
examples/nextjs-test/app/features/page.tsx
examples/nextjs-test/app/globals.css
examples/nextjs-test/app/layout.tsx
examples/nextjs-test/app/llms-full.txt/route.ts
examples/nextjs-test/app/llms.txt/route.ts
examples/nextjs-test/app/page.tsx
examples/nextjs-test/app/pricing/page.tsx
examples/nextjs-test/components/widgets.tsx
examples/nextjs-test/eslint.config.mjs
examples/nextjs-test/middleware.ts
examples/nextjs-test/next.config.ts
examples/nextjs-test/package.json
examples/nextjs-test/postcss.config.mjs
examples/nextjs-test/tsconfig.json
packages/core/README.md
packages/core/package.json
packages/core/src/generate.ts
packages/core/src/types.ts
packages/middleware/README.md
packages/middleware/package.json
packages/middleware/src/index.ts
packages/next/README.md
packages/next/package.json
packages/next/src/llms-full-txt-route.ts
packages/next/src/llms-txt-route.ts
packages/next/src/shared.ts
requirment.md

📝 Walkthrough

Walkthrough

This PR adds a complete Next.js example application demonstrating the @llmstxt package ecosystem, while enhancing core packages with URL-aware HTML-to-Markdown conversion and link resolution. The new example showcases LLMs.txt generation, full-content dumps, and markdown middleware for AI-friendly content serving. Core, middleware, and Next.js packages gain improved type safety, better documentation, and refined request handling.

Changes

Core Package Enhancements

Layer / File(s)	Summary
Type Signature Updates `packages/core/src/types.ts`	`GenerateLlmsFullTxtOptions.htmlToMarkdown` now accepts both `html` and `url` parameters for context-aware conversion.
Generator Implementation `packages/core/src/generate.ts`	New `extractHtmlTitle` helper; `generateLlmsFullTxt` now emits page metadata (title, url, description) and passes URL to htmlToMarkdown; pages include extracted title when available.
Documentation and Metadata `packages/core/README.md`, `package.json`	Expanded README with Node.js 18+ requirement, Express/Hono/Bun examples, comprehensive API tables, and Turndown-based markdown conversion guide; added `documentation` keyword and `prepare` script.

Middleware Package Enhancements

Layer / File(s)	Summary
Link Resolution `packages/middleware/src/index.ts`	New `resolveHref` helper converts relative URLs to absolute using page URL; `builtinHtmlToMarkdown` now accepts pageUrl and resolves anchor hrefs; middleware passes request URL to htmlToMarkdown.
Documentation and Metadata `packages/middleware/README.md`, `package.json`	Expanded README framing "Markdown for Agents" content negotiation, response headers table, "All options" reference, and Turndown example; added keywords and `prepare` script.

Next.js Package Enhancements

Layer / File(s)	Summary
Environment Resolution and Route Handlers `packages/next/src/shared.ts`, `llms-txt-route.ts`, `llms-full-txt-route.ts`	Development fallback to `http://localhost:3000` when `NODE_ENV=development`; route handlers switched from `NextResponse` to standard `Response` for HTTP compliance.
Documentation and Metadata `packages/next/README.md`, `package.json`	Expanded README with Next.js peer dependency, environment variables section with `.env.local` guidance, zero-config and options-based setup, caching details, and option tables; added keywords and `prepare` script.

Example Next.js Application

Layer / File(s)	Summary
Project Configuration `examples/nextjs-test/package.json`, `tsconfig.json`, `next.config.ts`, `postcss.config.mjs`, `eslint.config.mjs`, `.gitignore`	Initial setup with workspace dependencies on local `@llmstxt/*` packages, Next.js 16.2.6, React 19.2.4, Tailwind 4, ESLint 9, and TypeScript strict mode.
Global Styles and Root Layout `examples/nextjs-test/app/globals.css`, `app/layout.tsx`	CSS custom properties for light/dark theming; root layout with Geist fonts, metadata, and body element wrapping page children.
Reusable UI Components `examples/nextjs-test/components/widgets.tsx`	Six components: `Card`, `FeatureCard`, `PricingPlan` (with optional highlight), `CodeBlock`, `Navigation`, and `Hero` using Tailwind CSS.
Page Components `examples/nextjs-test/app/{about,blog,features,pricing,docs/getting-started,api-reference}/page.tsx`, `/page.tsx`	Eight pages demonstrating product messaging, feature showcase, pricing tiers, blog landing, API docs, getting-started guide, and home with environment display.
LLMs Route Handlers `examples/nextjs-test/app/{llms.txt,llms-full.txt}/route.ts`	Two route handlers wiring `createLlmsTxtHandler` and `createLlmsFullTxtHandler` with configured title and summary.
Middleware and Documentation `examples/nextjs-test/middleware.ts`, `README.md`, `AGENTS.md`, `CLAUDE.md`	Markdown middleware for `Accept: text/markdown` content negotiation; full example README with setup and test instructions; agent rules and cross-references.

Monorepo Documentation

Layer / File(s)	Summary
Root and Examples Docs `README.md`, `examples/README.md`, `requirment.md`	Root README expands with development fallback explanation and package install guidance; examples README introduces test projects and build/run instructions; obsolete requirment.md deleted.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐇 A shiny new Next.js app hops into view,
With pages and widgets and routing so true!
Core packages dance with URLs in hand,
Markdown middleware makes conversion grand.
The docs bloom with examples so clear—
LLMs love this fresh structure, my dear! ✨

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Muhammad-Hashim added 7 commits May 9, 2026 12:49

fix(next): use standard Response instead of NextResponse and add loca…

6a4cb1c

…lhost fallback - Replace NextResponse with native Response in route handlers to avoid importing next/server unnecessarily - Add localhost:3000 fallback in resolveBaseUrlFromEnv when NODE_ENV=development

build: add prepare scripts, keywords, and devDependencies to all pack…

551e85e

…ages - Add prepare script (npm run build) to core, middleware, and next packages - Add documentation, next-middleware, content-signal, route-handler keywords - Add @types/node devDependency to @llmstxt/next

chore: update package-lock.json and remove stale requirements file

0581745

Muhammad-Hashim merged commit 4906d96 into main May 9, 2026
3 of 4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: update package-lock.json and remove stale requirements file#3

chore: update package-lock.json and remove stale requirements file#3
Muhammad-Hashim merged 7 commits into
mainfrom
feat/add-new-package

Muhammad-Hashim commented May 9, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented May 9, 2026 •

edited

Loading

Review failed

Walkthrough

Changes

Estimated code review effort

Poem

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Muhammad-Hashim commented May 9, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of change

Pull-Request Checklist

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review failed

Walkthrough

Changes

Estimated code review effort

Poem

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Muhammad-Hashim commented May 9, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 9, 2026 •

edited

Loading