Commit bc6445c
feat(web): text file attachments for Ask (#1374)
* feat(web): add language model inputModalities capability plumbing
Add an optional `inputModalities` declaration to language model config and
expose a resolved capability set to the client.
- Schema: add optional `inputModalities` (`text` | `image` | `pdf`) to every
provider definition in `schemas/v3/languageModel.json` and regenerate the
schema types/snippets.
- Add a fail-closed `resolveModelInputModalities` resolver that defaults to
text-only when a model does not declare its input modalities.
- Expose the resolved `inputModalities` on the client-safe `LanguageModelInfo`
(populated via `getConfiguredLanguageModelsInfo` and the MCP ask path).
This is groundwork for chat file attachments. It adds no attachment UI and no
live provider capability probing yet.
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs: add CHANGELOG entry for language model inputModalities
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(schemas): split document types out of inputModalities
inputModalities now only enumerates true perceptual channels
(text | image | audio | video). Document/container formats like PDF
move to a separate fail-closed `supportedDocumentTypes` field, since
PDF is not a model modality but a format providers decompose into
text/image internally.
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs(schemas): clarify what counts as a document type
Tighten the inputModalities / supportedDocumentTypes descriptions to
remove the implication that omitting supportedDocumentTypes blocks all
non-text attachments. Clarify the taxonomy: single-medium files
(images, audio, video) and plain-text files (.txt, .md) are governed by
inputModalities; supportedDocumentTypes only gates rich compound
container formats like PDF.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): widen getLanguageModelKey param to keyable subset
LanguageModelInfo now has required inputModalities/supportedDocumentTypes,
so a raw LanguageModel config (where those are optional) is no longer
assignable to it. getLanguageModelKey only reads provider/model/displayName,
so type its parameter as that Pick subset, letting both LanguageModel and
LanguageModelInfo be keyed. Fixes the docker build type check.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(schemas,web): keep schema dist fresh and resolve types from source
Two dev-experience fixes for the stale-build-output footgun:
- schemas watch now runs `yarn build` (generate + tsc) instead of
generate-only, so editing a schema JSON during `yarn dev` refreshes
dist (both the .d.ts types and the runtime index.schema.js used by
ajv), not just the generated source.
- web tsconfig maps @sourcebot/schemas/v3|v2/* to the package source,
so type-checking and the IDE read committed source directly instead
of stale built .d.ts. Web only imports .type files (erased at
compile), so there is no bundling/runtime impact.
Co-authored-by: Cursor <cursoragent@cursor.com>
* First pass file attachments, picker and drag and drop, with preview cards, wired into user message via xml-like tags similar to system context
* escape key handle for modal and add missing description component
* refactor(web): resolve model capabilities from models.dev, not config.json
Re-source language model input-modality / document capabilities from the
models.dev catalog instead of hand-declared config.json fields, aligning
with the move to de-emphasize on-disk config in favor of automatic
resolution (the same catalog already backs context-window resolution).
- Revert the inputModalities/supportedDocumentTypes additions to
schemas/v3/languageModel.json and all regenerated artifacts; capabilities
are no longer declared in config.json.
- Extract the shared models.dev catalog plumbing (fetch/TTL/negative-cache/
stale-while-revalidate/provider-id overrides) into modelsDevCatalog.server.ts,
now consumed by both context-window and capability resolution.
- Add models.dev-backed resolveModelCapabilities (modelCapabilities.server.ts),
partitioning the catalog's modalities.input list into Sourcebot's
inputModalities (channels) and supportedDocumentTypes (containers); falls back
to text-only for uncatalogued / self-hosted models.
The client-safe LanguageModelInfo contract is unchanged; only the resolution
backend moved.
Co-authored-by: Cursor <cursoragent@cursor.com>
* paste-to-attachment handling, raw paste keychord enabled, toast with fallback
* stronger typing for contract
* remove blocking models.dev catalog request and add cache warm on startup
* cleanup warming
* remove button from toast
* add separate state to track pending attachments to avoid visual flicker of clear-on-submit
* explicitly import already hoisted uuid and change bad crypto call for uuid use
* add changelog entry
* remove granular file attachment controls in favor of single per-message cap of around 60-80k tokens
* pass attachments through the login/upgrade redirect
* persist a stable id on text attachments
Carry the pending attachment's client id through to the persisted message
instead of stripping it. This gives every text attachment a durable handle
from the moment the feature ships, so later attachment-referencing work has
no backwards-compatibility gap for attachments created before the field
existed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>1 parent 5d5a063 commit bc6445c
23 files changed
Lines changed: 936 additions & 58 deletions
File tree
- packages/web
- src
- app/(app)
- askgh/[owner]/[repo]/components
- chat
- components
- ee/features/chat
- components/chatThread
- features/chat
- components/chatBox
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
177 | 177 | | |
178 | 178 | | |
179 | 179 | | |
| 180 | + | |
180 | 181 | | |
181 | 182 | | |
182 | 183 | | |
| |||
200 | 201 | | |
201 | 202 | | |
202 | 203 | | |
| 204 | + | |
203 | 205 | | |
204 | 206 | | |
205 | 207 | | |
| |||
Lines changed: 13 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
| 8 | + | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
13 | | - | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
| 37 | + | |
36 | 38 | | |
37 | 39 | | |
38 | 40 | | |
| |||
67 | 69 | | |
68 | 70 | | |
69 | 71 | | |
70 | | - | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
71 | 77 | | |
72 | 78 | | |
73 | | - | |
74 | | - | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
75 | 82 | | |
76 | 83 | | |
77 | 84 | | |
| |||
107 | 114 | | |
108 | 115 | | |
109 | 116 | | |
110 | | - | |
| 117 | + | |
111 | 118 | | |
112 | 119 | | |
113 | 120 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
| |||
56 | 57 | | |
57 | 58 | | |
58 | 59 | | |
59 | | - | |
| 60 | + | |
60 | 61 | | |
61 | 62 | | |
62 | 63 | | |
| |||
92 | 93 | | |
93 | 94 | | |
94 | 95 | | |
95 | | - | |
| 96 | + | |
96 | 97 | | |
97 | 98 | | |
Lines changed: 44 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
Lines changed: 6 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| |||
31 | 32 | | |
32 | 33 | | |
33 | 34 | | |
| 35 | + | |
34 | 36 | | |
35 | 37 | | |
36 | 38 | | |
37 | 39 | | |
38 | 40 | | |
39 | 41 | | |
40 | | - | |
41 | | - | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
42 | 45 | | |
43 | 46 | | |
44 | 47 | | |
| |||
74 | 77 | | |
75 | 78 | | |
76 | 79 | | |
77 | | - | |
| 80 | + | |
78 | 81 | | |
79 | 82 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
105 | 105 | | |
106 | 106 | | |
107 | 107 | | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
108 | 115 | | |
109 | 116 | | |
110 | | - | |
| 117 | + | |
111 | 118 | | |
112 | 119 | | |
113 | 120 | | |
| |||
Lines changed: 13 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
| 7 | + | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
| 20 | + | |
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
| |||
72 | 73 | | |
73 | 74 | | |
74 | 75 | | |
| 76 | + | |
75 | 77 | | |
76 | 78 | | |
77 | 79 | | |
| |||
347 | 349 | | |
348 | 350 | | |
349 | 351 | | |
350 | | - | |
| 352 | + | |
351 | 353 | | |
352 | 354 | | |
353 | 355 | | |
354 | | - | |
| 356 | + | |
355 | 357 | | |
356 | 358 | | |
357 | 359 | | |
| |||
381 | 383 | | |
382 | 384 | | |
383 | 385 | | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
384 | 391 | | |
385 | 392 | | |
386 | 393 | | |
| |||
470 | 477 | | |
471 | 478 | | |
472 | 479 | | |
| 480 | + | |
473 | 481 | | |
474 | 482 | | |
475 | 483 | | |
| |||
520 | 528 | | |
521 | 529 | | |
522 | 530 | | |
| 531 | + | |
523 | 532 | | |
524 | 533 | | |
525 | 534 | | |
| |||
Lines changed: 26 additions & 18 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
| 13 | + | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
| |||
68 | 69 | | |
69 | 70 | | |
70 | 71 | | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
71 | 76 | | |
72 | 77 | | |
73 | 78 | | |
| |||
418 | 423 | | |
419 | 424 | | |
420 | 425 | | |
421 | | - | |
422 | | - | |
423 | | - | |
424 | | - | |
425 | | - | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
426 | 429 | | |
427 | | - | |
428 | | - | |
429 | | - | |
430 | | - | |
431 | | - | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
432 | 443 | | |
433 | 444 | | |
434 | 445 | | |
435 | | - | |
436 | | - | |
437 | | - | |
438 | | - | |
439 | | - | |
440 | | - | |
441 | | - | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
442 | 450 | | |
443 | 451 | | |
444 | 452 | | |
| |||
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
537 | 537 | | |
538 | 538 | | |
539 | 539 | | |
| 540 | + | |
540 | 541 | | |
541 | 542 | | |
542 | 543 | | |
| |||
0 commit comments