perf(analyses): optimize queries to reduce memory usage#322
perf(analyses): optimize queries to reduce memory usage#322sylvainkalache merged 1 commit intomainfrom
Conversation
Two optimizations in the get_analysis endpoint: 1. Select only email column from UserCorrelation instead of loading full objects (was loading all columns for 10K+ members when only email is needed) 2. Use load_only(id, uuid) on the 404 fallback query to avoid loading the potentially 30MB+ results JSON column just for an error message
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Claude finished @sylvainkalache's task —— View job PR Review: Query Optimization Analysis
|
Greptile OverviewGreptile SummaryThis PR implements two query optimizations to reduce memory usage in the
Key Changes
Issues Found
Confidence Score: 4/5
Important Files Changed
|
| member_emails = [ | ||
| row[0] for row in db.query(UserCorrelation.email).filter( | ||
| UserCorrelation.organization_id == analysis.organization_id, | ||
| UserCorrelation.organization_id.isnot(None), | ||
| UserCorrelation.email.isnot(None) | ||
| ).all() | ||
| ] |
There was a problem hiding this comment.
the new SQL-level null check won't filter empty string emails '' like the old Python-level if c.email check did. if empty emails exist in the DB, they'll now be included
| member_emails = [ | |
| row[0] for row in db.query(UserCorrelation.email).filter( | |
| UserCorrelation.organization_id == analysis.organization_id, | |
| UserCorrelation.organization_id.isnot(None), | |
| UserCorrelation.email.isnot(None) | |
| ).all() | |
| ] | |
| member_emails = [ | |
| row[0] for row in db.query(UserCorrelation.email).filter( | |
| UserCorrelation.organization_id == analysis.organization_id, | |
| UserCorrelation.organization_id.isnot(None), | |
| UserCorrelation.email.isnot(None), | |
| UserCorrelation.email != '' | |
| ).all() | |
| ] |
Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/app/api/endpoints/analyses.py
Line: 677:683
Comment:
the new SQL-level null check won't filter empty string emails `''` like the old Python-level `if c.email` check did. if empty emails exist in the DB, they'll now be included
```suggestion
member_emails = [
row[0] for row in db.query(UserCorrelation.email).filter(
UserCorrelation.organization_id == analysis.organization_id,
UserCorrelation.organization_id.isnot(None),
UserCorrelation.email.isnot(None),
UserCorrelation.email != ''
).all()
]
```
How can I resolve this? If you propose a fix, please make it concise.
Summary
Two query optimizations in the
get_analysisendpoint to reduce memory usage:.emailis needed)load_only(id, uuid)on the 404 fallback query to avoid loading the potentially 30MB+resultsJSON column just to build an error messageExpected Impact
Test plan