Skip to content

fix(xiaohongshu): improve search login-wall handling and detail output#298

Merged
jackwener merged 2 commits intojackwener:mainfrom
xiaokuili:fix/xiaohongshu-search-login-wall-and-detail-enrich
Mar 23, 2026
Merged

fix(xiaohongshu): improve search login-wall handling and detail output#298
jackwener merged 2 commits intojackwener:mainfrom
xiaokuili:fix/xiaohongshu-search-login-wall-and-detail-enrich

Conversation

@xiaokuili
Copy link
Contributor

@xiaokuili xiaokuili commented Mar 23, 2026

Description

Improve xiaohongshu search in two ways:

  1. Detect the Xiaohongshu search login wall and return an explicit error instead of silently returning [].
  2. Enrich search results with note detail data so the command returns:
    • a directly usable detail URL from the search result page
    • note content
    • comment count
    • top comments

This also fixes URL handling.
Previously the command could degrade result URLs into bare /explore/<noteId> links, which were not reliably reopenable.
Now it preserves the real detail link exposed by the search result page (search_result/...?...xsec_token=...), so the returned url can open the note detail page directly.

Related issue: #296

Type of Change

  • 🐛 Bug fix
  • ✨ New feature
  • 🌐 New site adapter
  • 📝 Documentation
  • ♻️ Refactor
  • 🔧 CI / build / tooling

Checklist

  • I ran the checks relevant to this PR
  • I updated tests or docs if needed
  • I included output or screenshots when useful

Documentation (if adding/modifying an adapter)

  • Added doc page under docs/adapters/ (if new adapter)
  • Updated docs/adapters/index.md table (if new adapter)
  • Updated sidebar in docs/.vitepress/config.mts (if new adapter)

Screenshots / Output

Checks run:

npx tsc --noEmit
npx vitest run src/

Added tests:

  • explicit login-wall error for xiaohongshu search
  • preserving the search_result detail URL and enriching rows with note detail fields

Before

{
  "rank": 2,
  "title": "某鱼买FSD被坑了4万",
  "author": "随风2025-10-10",
  "likes": "261",
  "url": "https://www.xiaohongshu.com/explore/68e90be80000000004022e66",
  "author_url": "https://www.xiaohongshu.com/user/profile/635a9c720000000018028b40?...",
  "content": "",
  "comment_count": "0",
  "comments": []
}

After

{
  "rank": 2,
  "title": "某鱼买FSD被坑了4万",
  "author": "随风",
  "likes": "261",
  "url": "https://www.xiaohongshu.com/search_result/68e90be80000000004022e66?xsec_token=...",
  "author_url": "https://www.xiaohongshu.com/user/profile/635a9c720000000018028b40?...",
  "content": "今天早上提车,昨天深夜,心血来潮搜了一下x鱼...",
  "comment_count": "302",
  "comments": [
    "KA330: 没有被坑啊。",
    "NONO: 你怎么敢某鱼花4.3W买的"
  ]
}

The After URL is taken from the real search-result detail link and can open the note detail page directly.

xiaokuili and others added 2 commits March 23, 2026 17:08
…, remove serial per-note enrichment

- Detect login wall and throw a clear error message (from original PR)
- Preserve search_result/ URL with xsec_token instead of degrading to /explore/<id>
- Add author_url to results
- Remove readNoteDetail() + sequential page.goto() per note (caused 60s+ delays
  for default limit=20 with 3s wait each)
- Simplify and unify DOM extraction logic (remove unused fallback anchor scan)
- Update tests: cover login-wall, URL preservation (assert single goto), and limit/filter
@jackwener jackwener merged commit f8bf663 into jackwener:main Mar 23, 2026
jackwener added a commit that referenced this pull request Mar 23, 2026
- docs/adapters/browser/xiaohongshu.md: fill in search command description
  (was empty), update usage examples with keyword positional arg
- TESTING.md: update unit test count 31→32 (search.test.ts added in #298),
  add xiaohongshu/search.test.ts to the adapter test file list
jackwener added a commit that referenced this pull request Mar 23, 2026
- docs/adapters/browser/xiaohongshu.md: fill in search command description
  (was empty), update usage examples with keyword positional arg
- TESTING.md: update unit test count 31→32 (search.test.ts added in #298),
  add xiaohongshu/search.test.ts to the adapter test file list
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants