[Due for payment 2026-01-20] [$250] Pass Page Context to Concierge from Side Panel

[thienlnam]

When a user chats with Concierge from the side panel (RHP), capture a simplified representation of the main app window content html and pass it as part of AddComment as a pageHTML parameter

1. When the user sends a message in the side panel from Concierge or the admins room
2. Capture simplified page html of the main body content (this would include things like semantic HTML elements (buttons, links, headings, menu items, inputs, etc.), visible text content from non-semantic containers, aria-label, href, alt, placeholder attributes where useful
3. Wrap the output with page URL <page url="/search?q=type:expense"> ... extracted content ... </page>
4. Pass to AddComment API. Add pageHTML parameter to addComment() function and only populate when isInSidePanel is true

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~022005826114541484946
• Upwork Job ID: 2005826114541484946
• Last Price Increase: 2025-12-30
• Automatic offers:
• QichenZhu | Reviewer | 109964002

Issue Owner

Current Issue Owner: @QichenZhu

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~022005826114541484946

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @QichenZhu (External)

[truph01]

🚨 Edited by proposal-police: This proposal was edited at 2025-12-30 03:19:12 UTC.

Proposal

Please re-state the problem that we are trying to solve in this issue.

• Pass Page Context to Concierge from Side Panel

What is the root cause of that problem?

• New feature

What changes do you think we should make in order to solve the problem?

1. Introduce a utility to extract simplified page HTML

   Create a function that capture simplified page html of the main body content (this would include things like semantic HTML elements (buttons, links, headings, menu items, inputs, etc.), visible text content from non-semantic containers, aria-label, href, alt, placeholder attributes where useful.

/**
 * Collect only meaningful HTML elements
 * Returns a minimal string representing page body
 */
function extractSimplifiedHTML() {
    const selector = `
        button,
        a[href],
        input,
        textarea,
        select,
        h1, h2, h3, h4, h5, h6,
        nav,
        ul, li,
        form,
        label
    `;

    // Query all meaningful elements
    const nodes = Array.from(document.body.querySelectorAll(selector));
    const simplifyNode = (node) => {
        const tag = node.tagName.toLowerCase();
        const attrs = {};

        // Capture useful attributes
        if (node.getAttribute('aria-label')) attrs['aria-label'] = node.getAttribute('aria-label');
        if (node.getAttribute('href')) attrs['href'] = node.getAttribute('href');
        if (node.getAttribute('placeholder')) attrs['placeholder'] = node.getAttribute('placeholder');
        if (node.getAttribute('alt')) attrs['alt'] = node.getAttribute('alt');

        // Visible text
        const text = node.textContent.trim();

        // Build opening tag
        let attrString = Object.entries(attrs)
            .map(([k, v]) => `${k}="${v.replace(/"/g, '"')}"`)
            .join(' ');

        if (attrString) attrString = ' ' + attrString;

        // Return minimal HTML
        return `<${tag}${attrString}>${text}</${tag}>`;
    };

    const simplified = nodes.map(n => simplifyNode(n)).join('\n');
    return `<page url="${document.location.href}">\n${simplified}\n</page>`;
}

• This function can be iterated on to better align with exactly what we want to send to the backend (e.g., filtering noise, deduplicating nodes, or truncating large outputs).
• This logic should only be applied on the web platform.

2. Pass page context when opening a report from the Side Panel

   • When calling addComment, include the extracted pageHTML payload when isInSidePanel === true.
   • This ensures Concierge receives page context only in scenarios where it is relevant and available.

3. In OP, it said "When the user sends a message in the side panel from Concierge or the admins room", so we can apply this change in admins room as well.

What alternative solutions did you explore? (Optional)

pageHTML

• Only include:

  • Tags: button, a[href], input, textarea, select,
    h1–h6, nav, ul, li, form, label
  • Attributes: aria-label, href, placeholder, alt
  • Text content

[ChinmayKarnik]

Proposal: Pass Page HTML Context to Concierge from Side Panel

Please re-state the problem that we are trying to solve in this issue

When users chat with Concierge from the side panel (RHP) while viewing any page in the app, Concierge needs context about what the user is looking at to provide better assistance. Currently, Concierge receives the user's message but has no information about the page content, making it difficult to provide contextual help.

What is the root cause of that problem?

This is a new feature. The current AddComment API flow doesn't:

• Know when a message is sent from the side panel vs. main chat
• Capture the main window's page content
• Send that context to the backend

The AddCommentOrAttachmentParams type has no field for page HTML context.

What changes do you think we should make in order to solve the problem?

1. Create PageContextUtils to capture and sanitize HTML (Web only)

File: src/libs/PageContextUtils.ts (NEW - 180 lines)

This utility captures semantic page structure while stripping all visual noise:

What gets KEPT:

• Semantic tags: button, a, input, textarea, select, h1-h6, nav, header, footer, main, form, label, ul, ol, li
• Essential attributes only: href, aria-label, placeholder, type, for
• Visible text content (direct text nodes only, no duplication)
• Page URL wrapped as: <page url="...">

What gets STRIPPED:

• ❌ ALL classes, IDs, inline styles
• ❌ ALL data-* attributes
• ❌ ALL SVG content and decorative attributes (fill, stroke, viewBox)
• ❌ ALL non-semantic elements (div, span - content extracted but tags removed)
• ❌ Password inputs (type="password")
• ❌ Hidden elements (display: none, visibility: hidden, opacity: 0)
• ❌ Side panel content ([data-testid="side-panel"])
• ❌ Modal/dialog content (role="dialog")

Key functions:

function capturePageHTMLContext(): string | undefined {
    // Only works on web
    if (typeof document === 'undefined') {
        return undefined;
    }
    
    const processNode = (node: Node): string => {
        // Recursive DOM traversal with filtering
        // Returns sanitized HTML string
    };
    
    const sanitized = processNode(document.body);
    const pageURL = window.location.pathname + window.location.search;
    
    // 50KB size limit with truncation
    if (sanitized.length > MAX_HTML_SIZE) {
        sanitized = sanitized.slice(0, MAX_HTML_SIZE) + '\n... [truncated]';
    }
    
    return `<page url="${pageURL}">\n${sanitized}\n</page>`;
}

Security features:

• Whitelist approach (only approved tags/attributes)
• Excludes passwords and hidden data
• Excludes side panel and modal content
• Size-limited to 50KB

2. Update API parameters

File: src/libs/API/parameters/AddCommentOrAttachmentParams.ts

type AddCommentOrAttachmentParams = {
    reportID: string;
    reportActionID?: string;
    commentReportActionID?: string | null;
    reportComment?: string;
    file?: FileObject;
    timezone?: string;
    clientCreatedTime?: string;
    isOldDotConciergeChat?: boolean;
    idempotencyKey?: string;
    pageHTML?: string; // ← Add this line
};

3. Modify Report.addActions to capture page context

File: src/libs/actions/Report.ts

Add imports:

import {Platform} from 'react-native';
import PageContextUtils from '@libs/PageContextUtils';

Update addActions() signature:

function addActions(
    reportID: string,
    notifyReportID: string,
    ancestors: Ancestor[],
    timezoneParam: Timezone,
    text = '',
    file?: FileObject,
    isInSidePanel = false, // ← Add this parameter
) {

Add capture logic (before API call):

// Capture page context only when:
// 1. On web platform
// 2. In side panel  
// 3. Chatting with Concierge or admin room
const report = allReports?.[`${ONYXKEYS.COLLECTION.REPORT}${reportID}`];
const shouldCapture = Platform.OS === 'web' 
    && isInSidePanel 
    && report
    && (ReportUtils.isConciergeChatReport(report) || ReportUtils.isAdminRoom(report));

const pageHTML = shouldCapture ? PageContextUtils.capturePageHTMLContext() : undefined;

// Add to parameters
const parameters: AddCommentOrAttachmentParams = {
    // ... existing fields ...
    ...(pageHTML && {pageHTML}), // Only include if captured
};

Update addComment() to accept and pass the flag:

function addComment(
    reportID: string,
    notifyReportID: string,
    ancestors: Ancestor[],
    text: string,
    timezoneParam: Timezone,
    shouldPlaySound?: boolean,
    isInSidePanel?: boolean, // ← Add this
) {
    if (shouldPlaySound) {
        playSound(SOUNDS.DONE);
    }
    addActions(reportID, notifyReportID, ancestors, timezoneParam, text, undefined, isInSidePanel);
}

4. Wire isInSidePanel through UI components

File: src/pages/home/ReportScreen.tsx

Pass isInSidePanel to ReportFooter:

<ReportFooter
    // ... other props ...
    isInSidePanel={isInSidePanel}
/>

File: src/pages/home/report/ReportFooter.tsx

Add to props type and pass to addComment:

type ReportFooterProps = {
    // ... existing props ...
    isInSidePanel?: boolean; // ← Add this
};

// In component body
const {isInSidePanel = false, /* other props */} = props;

// In onSubmitComment callback
addComment(
    targetReportID,
    report.reportID,
    targetReportAncestors,
    text,
    timezone,
    true,
    isInSidePanel, // ← Pass the flag
);

Example: Before vs After HTML

BEFORE (actual messy HTML with all noise):

<div class="default avatar" style="fill: #0891b2" data-testid="LHN-item">
  <svg width="40" height="40" viewBox="0 0 40 40" fill="currentColor">
    <path d="M20 0C8.95 0..." fill="#0891b2"/>
  </svg>
  <button class="flex items-center px-4 py-2" onclick="handleClick()" 
          data-testid="filter-button" role="button" tabindex="0"
          style="background-color: #0077B6">
    <span class="ml-2 text-sm font-medium">Filter expenses</span>
  </button>
</div>

AFTER (captured pageHTML - clean and minimal):

<button aria-label="Filter expenses">Filter expenses</button>

Size comparison:

• Before: ~450 bytes with noise
• After: ~63 bytes (86% reduction)
• Only semantic meaning preserved

What alternative solutions did you explore? (Optional)

Capturing full HTML with styles:
Rejected because it would create massive payloads (500KB+) and Concierge doesn't need visual styling, just semantic structure.

Capture when opening report:
The requirement says capture "when the user sends a message", not when they open. Users may send multiple messages, so addComment is the right trigger point.

Support native platforms:
Native doesn't have a DOM to query. Starting with web only makes sense since the requirement specifically mentions "page HTML". Can extend to native later with a different approach if needed.

Draft PR Link

Working implementation: #78625

Network Proof

Tested on web platform - pageHTML parameter successfully sent in API request:

The captured HTML is clean, semantic, and minimal - exactly what Concierge needs for contextual assistance.

[harjot96]

Proposal

1.Add pageHTML to AddComment payload (optional)
Add a new optional field: pageHTML?: string

This field is populated only for side panel messages to Concierge/Admins.
2) Web-only capture of “main app window” content
Implement a web-only DOM helper to extract a simplified representation of main content. The extraction:
Targets the main app DOM (#root) which excludes side panel UI (side panel is rendered via a web portal outside #root)
Picks the best “main content” subtree:
Prefer [data-testid="report-actions-view-wrapper"] (chat/report main pane)
Else prefer [role="main"]
Else prefer

Else fallback to largest visible child of #root
Extracts simplified HTML-like lines:
Maps ARIA roles to semantic tags when helpful (e.g. role="button" → )
Uses accessible name priority: aria-label → aria-labelledby → textContent
Avoids duplicated noise common in RN-web DOM by:
not counting text nodes that already sit inside a “semantic” ancestor
deduping identical output lines
Bounds and safety:
node and output size caps (prevents huge payloads)
excludes password values by design (never reads password input values)
3) Wire capture only into side panel send path
On submit:
If isInSidePanel AND room is Concierge/Admins:
compute pageHTML (web only)
pass into addComment(...) and include in AddComment API request
Otherwise:
do not send pageHTML
Implementation (What changed)
Code changes

1. AddComment params type extended
   File: src/libs/API/parameters/AddCommentOrAttachmentParams.ts
   Change: add optional pageHTML?: string to the request parameters type.
2. New helper to capture simplified page context (web only)
   File: src/libs/PageHTML/getSimplifiedMainPageHTML.ts
   New: DOM traversal + semantic extraction + wrapper ... .
3. Plumb pageHTML through addComment → addActions → API.write(AddComment)
   File: src/libs/actions/Report.ts
   Changes:
   addComment(...) signature extended to accept pageHTML?: string
   addActions(...) accepts pageHTML?: string
   pageHTML is only attached when the command is WRITE_COMMANDS.ADD_COMMENT (not attachment flows)
4. Capture and pass pageHTML only for side panel Concierge/Admins sends
   File: src/pages/home/report/ReportFooter.tsx
   Changes:
   Import getSimplifiedMainPageHTML
   On submit, if isInSidePanel and report is Concierge/Admins:
   capture pageHTML
   call addComment(..., pageHTML)

Branch
All changes are on branch: devo

[melvin-bot]

📣 @harjot96! 📣
Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
Please follow these steps:

1. Make sure you've read and understood the contributing guidelines.
2. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
3. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
4. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
   
   Format:

Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

[harjot96]

Contributor details

Your Expensify account email: twinkledhawan786@gmail.com
Upwork Profile Link: https://www.upwork.com/freelancers/twii

[melvin-bot]

✅ Contributor details stored successfully. Thank you for contributing to Expensify!

[yatin-italy]

Proposal

Problem
When users message Concierge from the Side Panel (RHP), agents lack context about the page the user is viewing. This often requires follow-up questions or screenshots.

Root Cause
Messages sent from the Side Panel do not include any representation of the main app view.

Solution
Attach a lightweight, semantic snapshot of the main application content when a message is sent from the Side Panel.

Implementation
Capture visible, meaningful elements (buttons, links, inputs, headings, labels) from the main app container.
Exclude all Side Panel content.
Prefer semantic text and accessibility attributes (aria-label, placeholder).
Truncate output and cap payload under 50KB.
Pass the snapshot as an optional pageHTML field in AddComment.
Populate pageHTML only for Side Panel messages in Concierge/Admin rooms.
Web-only (Platform.OS === 'web').

Reference Code
function getContextualMainContent() {
if (typeof document === 'undefined') return '';

const root =
    document.querySelector('[data-testid="main-content-wrapper"]') ||
    document.body;

const snapshot = [...root.querySelectorAll(
    'button,a,input,h1,h2,label,[role="button"]'
)]
    .filter(el =>
        (el.offsetWidth || el.offsetHeight || el.getClientRects().length) &&
        !el.closest('[data-testid="side-panel-wrapper"]')
    )
    .map(el => {
        const tag = el.tagName.toLowerCase();
        const label =
            el.getAttribute('aria-label') ||
            el.getAttribute('placeholder') ||
            '';
        const text =
            el.innerText?.trim().split('\n')[0].slice(0, 60) || '';

        return `<${tag} label="${label}">${text}</${tag}>`;
    })
    .join('\n')
    .slice(0, 48000);

return `<page url="${window.location.pathname + window.location.search}">

${snapshot}
`;
}