Skip to content

Fix: skip notes that fail to fetch instead of crashing#899

Open
lee90fly wants to merge 1 commit into
NanmiCoder:mainfrom
lee90fly:patch-1
Open

Fix: skip notes that fail to fetch instead of crashing#899
lee90fly wants to merge 1 commit into
NanmiCoder:mainfrom
lee90fly:patch-1

Conversation

@lee90fly
Copy link
Copy Markdown

Problem

When get_note_by_id_from_html returns empty for a single note (e.g. due to rate limit or anti-bot), the original code raises an Exception which crashes the entire crawler run.

Solution

Replace the raise Exception(...) with a utils.logger.warning(...) and return None, so the crawler skips that note and continues with the rest.

Test

Tested locally on xhs platform with 13 keywords. Previously crashed after 1-5 notes, now runs through full keyword list successfully.

Previously the crawler would crash on a single failed note. Now it logs a warning and continues.
@lee90fly lee90fly requested a review from NanmiCoder as a code owner May 18, 2026 21:27
@dosubot dosubot Bot added size:XS This PR changes 0-9 lines, ignoring generated files. bug Something isn't working labels May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant