Which is the correct BigQuery Partition Limit: 4000 or 10000? #152
Description
Course
data-engineering-zoomcamp
Question
Which is the correct BigQuery Partition Limit: 4000 or 10000?
Answer
In the playlist, it is stated that the limit was 4000. As of 2026, BigQuery has already increased this limit to 10000.
Context for Time-Based Partitioning: The practical impact depends on the partitioning granularity:
- Daily partitions (the most common) with the 10,000 limit cover over 27 years of data. The former 4,000 limit covered about 11 years.
- Hourly partitions with the 10,000 limit cover about 416 days (a little over a year). The former 4,000 limit covered roughly 166 days.
- Using the maximum limit of 10,000 partitions for a BigQuery table, a monthly partitioning strategy covers over
830 years of data.
Job Operations Limit: Separately, a single job operation (query or load) generally cannot modify or affect more than 4,000 partitions at once.
Checklist
- I have searched existing FAQs and this question is not already answered
- The answer provides accurate, helpful information
- I have included any relevant code examples or links