fix: clean up search indexing routes

[starc007]

Summary

• allow crawlers to access raw registry source while returning X-Robots-Tag: noindex
• redirect legacy component variants and renamed pages to their canonical docs pages
• redirect inferred nested source-file URLs to the component page that owns them
• preserve /api/og crawling because those image resources are intentionally referenced by page metadata

Validation

• bun run check
• bun test (22 passing)
• verified 76 redirect rules have no duplicates, valid-page collisions, or invalid destinations
• verified raw route returns 200, text/plain, and X-Robots-Tag: noindex