Skip to content

Harden ingestion workflow and add multi-document readiness #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Aryan1718 merged 5 commits into from
Jun 8, 2026
Merged

Harden ingestion workflow and add multi-document readiness #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
fec6015
Merge pull request #4 from Aryan1718/develop
Aryan1718 Jun 7, 2026
9fd2a85
Harden ingestion workflow and add multi-document readiness
varun3011 Jun 8, 2026
0409162
Show ingestion timing in dashboards
varun3011 Jun 8, 2026
bffd3ed
Fix security scan findings for ingestion load artifacts
varun3011 Jun 8, 2026
b0d4fd2
Fix frontend dependency vulnerabilities
varun3011 Jun 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ DAILY_TOKEN_LIMIT=100000
LOG_EACH_QUERY=false
RESERVATION_TTL_SECONDS=600
EMBEDDING_MODEL=text-embedding-3-small
MAX_QUERY_DOCUMENTS=10
MAX_QUERY_TOTAL_PAGES=100

# ===== Ingestion Limits =====
MAX_FILE_SIZE_BYTES=10485760
Expand All @@ -39,6 +41,9 @@ EMBEDDING_BATCH_SIZE=32
OPENAI_EMBEDDING_TIMEOUT_SECONDS=300
INGEST_EXTRACT_JOB_TIMEOUT_SECONDS=900
INGEST_INDEX_JOB_TIMEOUT_SECONDS=1800
INGEST_STALE_UPLOADED_SECONDS=3600
INGEST_STALE_EXTRACTING_SECONDS=1800
INGEST_STALE_INDEXING_SECONDS=3600

# ===== Client (React) =====
VITE_API_URL=http://localhost:8000
Expand Down
643 changes: 643 additions & 0 deletions artifacts/ingestion-load/ingestion-load-valid-10-20260608T081032Z.json
View file
Open in desktop

Large diffs are not rendered by default.

1,552 changes: 1,552 additions & 0 deletions artifacts/ingestion-load/ingestion-load-valid-25-20260608T081409Z.json
View file
Open in desktop

Large diffs are not rendered by default.

3,548 changes: 3,548 additions & 0 deletions artifacts/ingestion-load/ingestion-load-valid-50-20260608T083216Z.json
View file
Open in desktop

Large diffs are not rendered by default.

124 changes: 70 additions & 54 deletions client/package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions client/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,12 @@
"react-router-dom": "^6.21.3",
"@supabase/supabase-js": "^2.39.3",
"@tanstack/react-query": "^5.17.19",
"axios": "^1.6.5",
"axios": "^1.17.0",
"zustand": "^4.5.0",
"lucide-react": "^0.309.0",
"clsx": "^2.1.0",
"tailwindcss": "^3.4.1"
"tailwindcss": "^3.4.1",
"postcss": "^8.4.33"
},
"devDependencies": {
"@types/react": "^18.2.48",
Expand All @@ -29,7 +30,6 @@
"vite": "^5.0.11",
"vitest": "^1.2.0",
"eslint": "^8.56.0",
"autoprefixer": "^10.4.17",
"postcss": "^8.4.33"
"autoprefixer": "^10.4.17"
}
}
Loading
Loading