Skip to content

Commit 5dc8759

Browse files
calvarjorgecalvarjorge
authored
feat: lazy loading for Genie conversations (#163)
* feat: lazy loading for Genie conversations Load only the most recent page of messages when opening a conversation, and fetch older messages on demand when the user scrolls to the top. - Add `history_info` SSE event with pagination token - Add REST endpoint GET /:alias/conversations/:conversationId/messages - Frontend auto-triggers loading via IntersectionObserver with scroll position preservation - Consolidate to single `listConversationMessages` method in connector Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * feat: lazy loading for Genie conversations Load only the most recent page of messages when opening a conversation, and fetch older messages on demand when the user scrolls to the top. - Add `history_info` SSE event with pagination token to existing getConversation endpoint (no new endpoints) - Frontend auto-triggers loading via IntersectionObserver with scroll position preservation using the same SSE endpoint with pageToken - Consolidate to single `listConversationMessages` method in connector Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * refactor: extract fetchConversationPage and fix review issues - Extract shared fetchConversationPage for loadHistory and loadOlderMessages - Rename processEvent to processStreamEvent (only used for sendMessage) - Pass stable React setters directly instead of recreating paginationCallbacks - Add abort signal to loadOlderMessages to prevent leaks on unmount Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * fix: use genieConnectorDefaults.initialPageSize in tests Reference the constant directly instead of hardcoding the value, preventing test failures when the default is changed. Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * refactor: extract scroll hooks from GenieChatMessageList Extract useScrollManagement and useLoadOlderOnScroll into custom hooks, inline StreamingIndicator, leaving the component as clean JSX. Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * refactor: minor cleanup in GenieChatMessageList - useLoadOlderOnScroll owns its sentinel ref internally - Simplify message rendering with filter+map Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * refactor: simplify useGenieChat internals - Merge ref-sync effects into one - Extract fetchPage helper shared by loadHistory and loadOlderMessages - Simplify query_result handler with .map() instead of imperative loop Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * fix: address PR review feedback - Rename hasOlderMessages/loadOlderMessages to hasPreviousPage/ fetchPreviousPage following TanStack Query conventions, add isFetchingPreviousPage - Use separate AbortController for pagination to avoid aborting in-progress streams - Fix Express query param type safety (typeof check vs as cast) - Add test for paginated request failure Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * docs: re-generate docs after change * fix: use typeof check for requestId in sendMessage handler Consistent with getConversation handler — avoids unsafe as string cast for Express query params. Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * fix: scroll management races and observer rapid-fire loop - useScrollManagement no longer fires on status-only changes, preventing scroll-to-bottom yank when "loading-older" spinner appears - sendMessage aborts in-flight pagination to prevent status stomping - fetchPreviousPage guards setStatus to not overwrite concurrent streams - IntersectionObserver skips initial callback after re-subscription to prevent rapid-fire pagination loop Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * fix: pagination error handling, type safety, and performance - Server-sent error events during pagination now set error status (hadError flag prevents status flashing to idle) - Replace unsafe MutableRefObject cast with plain object type in fetchPage - Restore reverse-scan-with-break for query_result in processStreamEvent - Add ResizeObserver to keep scroll height ref fresh for async content Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * fix: simplify error handling, remove hadError flag - Set error status directly in onError callback instead of threading hadError flag through return value - Remove freeze/unfreeze scroll momentum plumbing (not effective enough) - Clean up unused code Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * fix: abort pagination on conversation switch, arm observer via rAF - loadHistory aborts in-flight pagination to prevent stale messages from old conversation being prepended - Replace initialFire boolean with requestAnimationFrame arming so short conversations where sentinel is genuinely visible still trigger loading automatically Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * fix: enforce minimum delay before prepending older messages Add MIN_PREVIOUS_PAGE_LOAD_MS to ensure scroll inertia fully settles before fetchPreviousPage prepends older messages to the chat. Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> * fix: prevent scroll jump when loading-older indicator disappears The useScrollManagement hook's useLayoutEffect only depended on messages.length, so prevScrollHeightRef was never updated when the loading indicator appeared. Adding status to the dependency array lets the effect capture the correct scrollHeight before messages are prepended, producing an accurate delta. Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com> --------- Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
1 parent b9a6bd8 commit 5dc8759

10 files changed

Lines changed: 598 additions & 152 deletions

File tree

‎docs/docs/api/appkit-ui/genie/GenieChatMessageList.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,8 @@ Scrollable message list that renders Genie chat messages with auto-scroll, skele
1818
| `messages` | `GenieMessageItem[]` | ✓ | - | Array of messages to display |
1919
| `status` | `enum` | ✓ | - | Current chat status (controls loading indicators and skeleton placeholders) |
2020
| `className` | `string` | | - | Additional CSS class for the scroll area |
21+
| `hasPreviousPage` | `boolean` | | `false` | Whether a previous page of older messages exists |
22+
| `onFetchPreviousPage` | `(() => void)` | | - | Callback to fetch the previous page of messages |
2123

2224

2325

‎packages/appkit-ui/src/react/genie/genie-chat-message-list.tsx‎

Lines changed: 154 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { useEffect, useRef } from "react";
1+
import { useEffect, useLayoutEffect, useRef } from "react";
22
import { cn } from "../lib/utils";
33
import { ScrollArea } from "../ui/scroll-area";
44
import { Skeleton } from "../ui/skeleton";
@@ -13,6 +13,10 @@ export interface GenieChatMessageListProps {
1313
status: GenieChatStatus;
1414
/** Additional CSS class for the scroll area */
1515
className?: string;
16+
/** Whether a previous page of older messages exists */
17+
hasPreviousPage?: boolean;
18+
/** Callback to fetch the previous page of messages */
19+
onFetchPreviousPage?: () => void;
1620
}
1721

1822
const STATUS_LABELS: Record<string, string> = {
@@ -26,41 +30,158 @@ function formatStatus(status: string): string {
2630
return STATUS_LABELS[status] ?? status.replace(/_/g, " ").toLowerCase();
2731
}
2832

29-
function StreamingIndicator({ messages }: { messages: GenieMessageItem[] }) {
30-
const last = messages[messages.length - 1];
31-
if (last?.role === "assistant" && last.id === "") {
32-
return (
33-
<div className="flex items-center gap-2 text-sm text-muted-foreground px-11">
34-
<Spinner className="h-3 w-3" />
35-
<span>{formatStatus(last.status)}</span>
36-
</div>
33+
function getViewport(scrollRef: React.RefObject<HTMLDivElement | null>) {
34+
return scrollRef.current?.querySelector<HTMLElement>(
35+
'[data-slot="scroll-area-viewport"]',
36+
);
37+
}
38+
39+
/**
40+
* Manages scroll position: scrolls to bottom on append/initial load,
41+
* preserves position when older messages are prepended.
42+
*/
43+
function useScrollManagement(
44+
scrollRef: React.RefObject<HTMLDivElement | null>,
45+
messages: GenieMessageItem[],
46+
status: GenieChatStatus,
47+
) {
48+
const prevFirstMessageIdRef = useRef<string | null>(null);
49+
const prevScrollHeightRef = useRef(0);
50+
const prevMessageCountRef = useRef(0);
51+
52+
// Keep prevScrollHeightRef fresh when async content (images, embeds)
53+
// changes the viewport height between renders.
54+
useEffect(() => {
55+
const viewport = getViewport(scrollRef);
56+
if (!viewport) return;
57+
58+
const observer = new ResizeObserver(() => {
59+
prevScrollHeightRef.current = viewport.scrollHeight;
60+
});
61+
observer.observe(viewport);
62+
return () => observer.disconnect();
63+
}, [scrollRef]);
64+
65+
// biome-ignore lint/correctness/useExhaustiveDependencies: react to message count AND status so prevScrollHeightRef stays accurate when the loading indicator appears/disappears
66+
useLayoutEffect(() => {
67+
const viewport = getViewport(scrollRef);
68+
if (!viewport) return;
69+
70+
const count = messages.length;
71+
const countChanged = count !== prevMessageCountRef.current;
72+
prevMessageCountRef.current = count;
73+
74+
// Nothing to do if message count didn't change (e.g. status-only transition)
75+
if (!countChanged) {
76+
prevScrollHeightRef.current = viewport.scrollHeight;
77+
return;
78+
}
79+
80+
const firstMessageId = messages[0]?.id ?? null;
81+
const wasPrepend =
82+
prevFirstMessageIdRef.current !== null &&
83+
firstMessageId !== prevFirstMessageIdRef.current;
84+
85+
if (wasPrepend && prevScrollHeightRef.current > 0) {
86+
// Older messages prepended — preserve scroll position
87+
const delta = viewport.scrollHeight - prevScrollHeightRef.current;
88+
viewport.scrollTop += delta;
89+
} else {
90+
// Messages appended or initial load — scroll to bottom
91+
viewport.scrollTop = viewport.scrollHeight;
92+
}
93+
94+
prevFirstMessageIdRef.current = firstMessageId;
95+
prevScrollHeightRef.current = viewport.scrollHeight;
96+
}, [messages.length, status]);
97+
}
98+
99+
/**
100+
* Observes a sentinel element at the top of the scroll area and triggers
101+
* `onFetchPreviousPage` when the user scrolls to the top (only if content overflows).
102+
* Returns a ref to attach to the sentinel element.
103+
*/
104+
function useLoadOlderOnScroll(
105+
scrollRef: React.RefObject<HTMLDivElement | null>,
106+
shouldObserve: boolean,
107+
onFetchPreviousPage?: () => void,
108+
) {
109+
const sentinelRef = useRef<HTMLDivElement>(null);
110+
const onFetchPreviousPageRef = useRef(onFetchPreviousPage);
111+
onFetchPreviousPageRef.current = onFetchPreviousPage;
112+
113+
useEffect(() => {
114+
const sentinel = sentinelRef.current;
115+
const viewport = getViewport(scrollRef);
116+
if (!sentinel || !viewport || !shouldObserve) return;
117+
118+
// The observer fires synchronously on observe() if the sentinel is
119+
// already visible. We arm it on the next frame so that synchronous
120+
// initial fire is ignored, but a real intersection (user genuinely
121+
// at the top on a short conversation) triggers on subsequent frames.
122+
let armed = false;
123+
const frameId = requestAnimationFrame(() => {
124+
armed = true;
125+
});
126+
127+
const observer = new IntersectionObserver(
128+
(entries) => {
129+
if (!armed) return;
130+
const isScrollable = viewport.scrollHeight > viewport.clientHeight;
131+
if (entries[0]?.isIntersecting && isScrollable) {
132+
onFetchPreviousPageRef.current?.();
133+
}
134+
},
135+
{ root: viewport, threshold: 0 },
37136
);
38-
}
39-
return null;
137+
138+
observer.observe(sentinel);
139+
return () => {
140+
cancelAnimationFrame(frameId);
141+
observer.disconnect();
142+
};
143+
}, [scrollRef, shouldObserve]);
144+
145+
return sentinelRef;
40146
}
41147

42148
/** Scrollable message list that renders Genie chat messages with auto-scroll, skeleton loaders, and a streaming indicator. */
43149
export function GenieChatMessageList({
44150
messages,
45151
status,
46152
className,
153+
hasPreviousPage = false,
154+
onFetchPreviousPage,
47155
}: GenieChatMessageListProps) {
48156
const scrollRef = useRef<HTMLDivElement>(null);
49157

50-
// Scroll only the ScrollArea viewport, not the page
51-
// biome-ignore lint/correctness/useExhaustiveDependencies: intentional triggers for auto-scroll
52-
useEffect(() => {
53-
const viewport = scrollRef.current?.querySelector<HTMLElement>(
54-
'[data-slot="scroll-area-viewport"]',
55-
);
56-
if (viewport) {
57-
viewport.scrollTop = viewport.scrollHeight;
58-
}
59-
}, [messages.length, status]);
158+
const sentinelRef = useLoadOlderOnScroll(
159+
scrollRef,
160+
hasPreviousPage && status !== "loading-older",
161+
onFetchPreviousPage,
162+
);
163+
useScrollManagement(scrollRef, messages, status);
164+
165+
const lastMessage = messages[messages.length - 1];
166+
const showStreamingIndicator =
167+
status === "streaming" &&
168+
lastMessage?.role === "assistant" &&
169+
lastMessage.id === "";
60170

61171
return (
62172
<ScrollArea ref={scrollRef} className={cn("flex-1 min-h-0 p-4", className)}>
63173
<div className="flex flex-col gap-4">
174+
{hasPreviousPage && <div ref={sentinelRef} className="h-px" />}
175+
176+
{status === "loading-older" && (
177+
<div className="flex items-center justify-center gap-2 py-2">
178+
<Spinner className="h-3 w-3" />
179+
<span className="text-sm text-muted-foreground">
180+
Loading older messages...
181+
</span>
182+
</div>
183+
)}
184+
64185
{status === "loading-history" && messages.length === 0 && (
65186
<div className="flex flex-col gap-4">
66187
<Skeleton className="h-12 w-3/4" />
@@ -69,15 +190,19 @@ export function GenieChatMessageList({
69190
</div>
70191
)}
71192

72-
{messages.map((msg) => {
73-
if (msg.role === "assistant" && msg.id === "" && !msg.content) {
74-
return null;
75-
}
76-
return <GenieChatMessage key={msg.id} message={msg} />;
77-
})}
193+
{messages
194+
.filter(
195+
(msg) => msg.role !== "assistant" || msg.id !== "" || msg.content,
196+
)
197+
.map((msg) => (
198+
<GenieChatMessage key={msg.id} message={msg} />
199+
))}
78200

79-
{status === "streaming" && messages.length > 0 && (
80-
<StreamingIndicator messages={messages} />
201+
{showStreamingIndicator && (
202+
<div className="flex items-center gap-2 text-sm text-muted-foreground px-11">
203+
<Spinner className="h-3 w-3" />
204+
<span>{formatStatus(lastMessage.status)}</span>
205+
</div>
81206
)}
82207

83208
{messages.length === 0 && status === "idle" && (

‎packages/appkit-ui/src/react/genie/genie-chat.tsx‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,15 @@ export function GenieChat({
1212
placeholder,
1313
className,
1414
}: GenieChatProps) {
15-
const { messages, status, error, sendMessage, reset } = useGenieChat({
15+
const {
16+
messages,
17+
status,
18+
error,
19+
sendMessage,
20+
reset,
21+
hasPreviousPage,
22+
fetchPreviousPage,
23+
} = useGenieChat({
1624
alias,
1725
basePath,
1826
});
@@ -32,7 +40,12 @@ export function GenieChat({
3240
</div>
3341
)}
3442

35-
<GenieChatMessageList messages={messages} status={status} />
43+
<GenieChatMessageList
44+
messages={messages}
45+
status={status}
46+
hasPreviousPage={hasPreviousPage}
47+
onFetchPreviousPage={fetchPreviousPage}
48+
/>
3649

3750
{error && (
3851
<div className="shrink-0 px-4 py-2 text-sm text-destructive bg-destructive/10 border-t">

‎packages/appkit-ui/src/react/genie/types.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ export type {
99
export type GenieChatStatus =
1010
| "idle"
1111
| "loading-history"
12+
| "loading-older"
1213
| "streaming"
1314
| "error";
1415

@@ -40,6 +41,12 @@ export interface UseGenieChatReturn {
4041
error: string | null;
4142
sendMessage: (content: string) => void;
4243
reset: () => void;
44+
/** Whether a previous page of older messages exists */
45+
hasPreviousPage: boolean;
46+
/** Whether a previous page is currently being fetched */
47+
isFetchingPreviousPage: boolean;
48+
/** Fetch the previous page of older messages */
49+
fetchPreviousPage: () => void;
4350
}
4451

4552
export interface GenieChatProps {

0 commit comments

Comments
 (0)