⚡️ Speed up function _mark_non_table_inferred_for_removal_if_has_subregion_relationship by 114%
#47
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📄 114% (1.14x) speedup for
_mark_non_table_inferred_for_removal_if_has_subregion_relationshipinunstructured/partition/pdf_image/pdfminer_processing.py⏱️ Runtime :
16.7 milliseconds→7.78 milliseconds(best of86runs)📝 Explanation and details
The optimization replaces the original two separate function calls to
bboxes1_is_almost_subregion_of_bboxes2()and the underlyingareas_of_boxes_and_intersection_area()with a single, fused Numba-compiled function_areas_and_subregion_mask().Key Changes:
@njit(cache=True, fastmath=True)decorator compiles the intersection area computation and subregion logic to optimized machine code, eliminating Python interpretation overheadinter_area,boxa_area, andboxb_areathat consumed memory and required additional vectorized operationsWhy It's Faster:
The original implementation had two expensive calls (82% and 37% of runtime respectively in the line profiler) that involved:
get_coords_from_bboxes()The Numba version eliminates the NumPy vectorization overhead by using explicit nested loops that compile to efficient machine code, avoiding temporary array allocations and reducing memory bandwidth requirements.
Impact on Workloads:
Based on the function reference, this optimization directly benefits PDF layout merging operations in
array_merge_inferred_layout_with_extracted_layout(), which is a core function for document processing. The 114% speedup is particularly valuable for:The optimization shows consistent 200-800% improvements across all test scenarios, with the greatest benefits on larger datasets where the O(N×M) complexity of comparing all inferred vs extracted elements becomes most expensive.
✅ Correctness verification report:
🌀 Generated Regression Tests and Runtime
To edit these changes
git checkout codeflash/optimize-_mark_non_table_inferred_for_removal_if_has_subregion_relationship-mjddnijpand push.