Skip to content

feat: add Tavily-powered Reddit search alongside BrightData pipeline#2

Open
tavily-integrations wants to merge 2 commits into
1rishu0:mainfrom
Tavily-FDE:feat/tavily-migration/tavily-reddit-parallel
Open

feat: add Tavily-powered Reddit search alongside BrightData pipeline#2
tavily-integrations wants to merge 2 commits into
1rishu0:mainfrom
Tavily-FDE:feat/tavily-migration/tavily-reddit-parallel

Conversation

@tavily-integrations
Copy link
Copy Markdown

Summary

Adds a fast Tavily-powered Reddit search (site:reddit.com via include_domains) as a parallel, lightweight alternative to the existing BrightData snapshot-based Reddit pipeline. The BrightData pipeline (trigger → poll → download) remains fully intact; the Tavily path provides quicker results without snapshot latency.

Changes

web_operations.py

  • Added tavily_reddit_search(query) function using TavilyClient with include_domains=["reddit.com"] and search_depth="advanced"
  • Returns results in the same parsed_posts format as reddit_search_api()

main.py

  • Added tavily_reddit_results and tavily_reddit_analysis fields to State TypedDict
  • Added tavily_reddit_search_node and analyze_tavily_reddit_results graph nodes
  • Wired tavily_reddit_search from START in parallel with existing search nodes
  • Connected analyze_tavily_reddit_resultssynthesize_analyses
  • Updated synthesize_analyses to include tavily_reddit_analysis
  • Updated initial state dict and launch message

prompts.py

  • Added tavily_reddit_analysis_system() and tavily_reddit_analysis_user() prompt templates
  • Added get_tavily_reddit_analysis_messages() convenience function
  • Updated synthesis_user() and get_synthesis_messages() to accept and include tavily_reddit_analysis
  • Updated synthesis system prompt to reference Tavily Reddit source

pyproject.toml

  • Added tavily-python>=0.5.0 dependency

.gitignore (new)

  • Added __pycache__/ and *.pyc

Environment Variable Changes

  • Requires TAVILY_API_KEY environment variable (shared with other Tavily migration units)

Notes for Reviewers

  • The BrightData Reddit pipeline is completely untouched — this is purely additive
  • The Tavily path runs in parallel from START, so it does not add latency to the existing pipeline
  • snapshot_operations.py was not modified

Automated Review

  • Passed after 2 attempt(s)
  • Final review: The tavily-reddit-parallel unit correctly adds a Tavily-based Reddit search as a parallel path in the LangGraph agent. All five issues from the first review attempt have been addressed: content field is present in parsed results, early return guard exists in analyze_tavily_reddit_results, trailing newlines are added, and .gitignore is confirmed from the prerequisite unit. The graph wiring, prompt templates, dependency declaration, and graceful API key failure handling are all correct. One minor pre-existing type annotation inconsistency is noted but follows the existing codebase pattern.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant