feat(verifier): inject YouTube URLs as FileData for native video understanding

[MrOrz]

Summary

Cofacts.ai often hallucinates or provides incomplete context when dealing with YouTube videos because it previously only relied on web metadata. This PR enables native video understanding for YouTube URLs across the agent system.

YouTube Native Video Understanding

• Automatic Injection: Added inject_youtube_filedata as a before_model_callback for both ai_verifier and ai_investigator. It scans for YouTube URLs in the user messages and appends them as FileData parts, allowing Gemini to watch the videos directly.
• Complementary Context: Original URLs remain intact, so url_context (and Google Search grounding) continues to provide page-level metadata (titles, upload dates, descriptions) alongside the video frames.
• Strict Instructions (Hard Rules): Updated instructions for verifier and investigator to enforce a "No Training Knowledge" policy for media. Agents are now required to report only what is directly visible/audible to prevent hallucinating identities or event details not present in the video.

System Robustness & Upgrades

• Model Stability: Upgraded all proofreader agents from gemini-3.1-flash-lite-preview to the stable gemini-3.1-flash-lite.
• Error Handling: Implemented handle_writer_tool_error via on_tool_error_callback for the ai_writer. This prevents the entire session from crashing if a tool (like Google Search or url_context) fails, allowing the writer to react to the error message instead.

Architecture

writer → investigator / verifier (input with youtube url)
              ↓ before_model_callback
              detect youtube urls in llm_request.contents
              append Content(role="user", parts=[FileData(file_uri=url)])
              ↓ model call
              url_context → youtube page HTML metadata (title, description)
              FileData    → Gemini watches video content directly

Test Plan

https://langfuse.cofacts.tw/project/cmm0emerr0001qi07eugd0760/traces/3c3c0aa36707f8a1a05f2487a5fca052?timestamp=2026-05-30T07:29:47.923Z

• Verified that YouTube URLs are correctly injected as FileData in LLM requests.
• Confirmed ai_verifier reports both page metadata (via url_context) and observable video content.
• Confirmed that regular web URLs still work as expected without FileData injection.
• Verify ai_writer gracefully handles tool errors in a live session.

---

Generated by Gemini CLI

[gemini-code-assist]

Code Review

This pull request introduces the ability for the ai_verifier agent to process YouTube video content by extracting URLs from the conversation and appending them as FileData to the LLM request. Feedback focuses on expanding the YouTube URL regex to include shorts and live streams, ensuring the LLM request structure remains valid by avoiding consecutive user messages, and adding a missing type hint for the llm_request parameter.

[MrOrz]

@claude review

[claude]

Claude finished @MrOrz's task in 2m 26s —— View job

---

Code Review

I've completed a thorough review of this PR. Overall, the implementation is solid and the YouTube video understanding feature is well-designed. I found a few areas that could be improved for robustness and clarity.

---

🔍 Findings

1. Missing Error Handling in inject_youtube_filedata (High Priority)

File: adk/cofacts_ai/agent.py:216-242

The callback function lacks error handling. If any exception occurs during URL extraction or FileData construction, it could crash the entire agent invocation.

Issue: Operations like accessing part.text, regex matching, or constructing FileData could fail.

Recommendation: Wrap the function body in a try-except block:

def inject_youtube_filedata(
    callback_context: CallbackContext, llm_request: Any
) -> None:
    """Before-model callback for ai_investigator and ai_verifier.
    
    For each user message that contains YouTube URLs, appends FileData parts
    into the same parts array so Gemini can watch the videos inline.
    The original URLs are kept intact so url_context still fetches their
    title/description metadata.
    
    Ref: https://ai.google.dev/gemini-api/docs/video-understanding#youtube
    """
    try:
        for content in llm_request.contents:
            if content.role != "user" or not content.parts:
                continue
            youtube_urls = []
            for part in content.parts:
                if part.text:
                    youtube_urls.extend(_YOUTUBE_URL_RE.findall(part.text))
            seen = set()
            for url in youtube_urls:
                if url not in seen:
                    seen.add(url)
                    content.parts.append(
                        genai_types.Part(file_data=genai_types.FileData(file_uri=url))
                    )
    except Exception as e:
        # Log but don't crash - YouTube injection is an enhancement, not critical
        # In production, you might want to use proper logging here
        pass
    return None

Fix this →

---

2. Deduplication Scope Consideration (Medium Priority)

File: adk/cofacts_ai/agent.py:235-241

The current implementation deduplicates YouTube URLs within each content block separately. If the same URL appears in multiple user messages within a single request, it will be injected multiple times.

Current behavior:

for content in llm_request.contents:  # Each content gets its own `seen` set
    # ...
    seen = set()
    for url in youtube_urls:
        if url not in seen:
            seen.add(url)
            # ...

Question: Is this intentional?

• If yes: Document why each content block should get separate copies
• If no: Move seen = set() outside the outer loop:

seen = set()  # Deduplicate across entire request
for content in llm_request.contents:
    if content.role != "user" or not content.parts:
        continue
    # ... rest of logic

This could improve performance and reduce token usage if the same video appears multiple times.

---

3. Type Annotation Completeness (Low Priority)

File: adk/cofacts_ai/agent.py:216-218

The function signature could be more explicit:

Current:

def inject_youtube_filedata(
    callback_context: CallbackContext, llm_request: Any
) -> None:

Suggested (if the ADK provides a type for the request object):

from google.genai.types import GenerateContentRequest  # or appropriate type

def inject_youtube_filedata(
    callback_context: CallbackContext, 
    llm_request: GenerateContentRequest
) -> None:

Using Any is acceptable if the ADK doesn't export the proper type, but worth checking the ADK documentation.

---

4. Instruction Clarity in Verifier (Low Priority)

File: adk/cofacts_ai/agent.py:307-311

The instruction says url_context is "MANDATORY" but could be clearer about whether this applies to ALL URLs or just certain types:

1. Call url_context for ALL provided URLs in one call (up to 20) — this is MANDATORY.
   url_context fetches the web PAGE metadata (title, publish date, description), which is
   always required regardless of what else is visible in this conversation.
   Note: for video URLs (e.g. YouTube), page metadata and video frames are complementary —
   url_context gives you the upload date; the video gives you observable content.

Suggestion: Make it explicit that ALL URLs need url_context:

1. Call url_context for ALL provided URLs (up to 20 per call) — this is MANDATORY, even for video URLs.
   url_context fetches web PAGE metadata (title, publish date, description) from the HTML.
   For video URLs like YouTube, page metadata and video frames are complementary:
   - url_context → upload date, uploader name, page title/description
   - FileData → observable video content (speech, visuals, on-screen text)

---

✅ What Looks Good

1. YouTube URL Regex - Comprehensive pattern covering all major YouTube URL formats after previous review feedback
2. Error Handling for Writer - The handle_writer_tool_error callback is well-designed and provides clear error messages
3. Model Upgrade - Moving from preview to stable model (gemini-3.1-flash-lite) is the right call
4. Three-Layer Reporting - The structured approach for video verification (page metadata → title/description → observable content) is excellent for preventing hallucination
5. No Training Knowledge Rule - Strong safeguard against false identification in videos
6. Architecture - Using before_model_callback is the right approach for this feature

---

📊 Overall Assessment

Strengths:

• Solves a real problem (YouTube video hallucination)
• Clean architectural approach
• Comprehensive instructions for agents
• Good separation of concerns

Risks:

• Missing error handling could cause failures (should fix)
• Potential redundant processing if same URL appears multiple times (worth investigating)

Recommendation: Add error handling (#1), clarify deduplication intent (#2), then this is good to merge.

---

📸 Test Plan Review

The test plan looks solid:

• ✅ FileData injection verified
• ✅ Verifier reports both metadata and video content
• ✅ Regular URLs still work
• ⏳ Writer tool error handling pending

Consider adding a test case for:

• Multiple YouTube URLs in a single message
• Same YouTube URL appearing multiple times
• Malformed YouTube URLs that match the regex

---

[MrOrz]

Verified