Skip to content

improvement: Bigtable to Parquet Template config options#3782

Open
brandtnewton wants to merge 4 commits into
GoogleCloudPlatform:mainfrom
brandtnewton:timeout-fix
Open

improvement: Bigtable to Parquet Template config options#3782
brandtnewton wants to merge 4 commits into
GoogleCloudPlatform:mainfrom
brandtnewton:timeout-fix

Conversation

@brandtnewton
Copy link
Copy Markdown

Some users have requested the ability to configure longer read row timeouts for the Bigtable to Parquet Template.

@brandtnewton brandtnewton force-pushed the timeout-fix branch 2 times, most recently from 8dec2f5 to cb9982e Compare May 11, 2026 14:21
@brandtnewton brandtnewton changed the title feat: Bigtable to Parquet Template config options improvement: Bigtable to Parquet Template config options May 11, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 11, 2026

Codecov Report

❌ Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.48%. Comparing base (4d5e027) to head (3cafa23).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
...gle/cloud/teleport/bigtable/BigtableToParquet.java 0.00% 3 Missing ⚠️

❌ Your patch check has failed because the patch coverage (0.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@              Coverage Diff              @@
##               main    #3782       +/-   ##
=============================================
+ Coverage     33.47%   53.48%   +20.01%     
- Complexity      482     6245     +5763     
=============================================
  Files           215     1085      +870     
  Lines         12891    66446    +53555     
  Branches       1262     7399     +6137     
=============================================
+ Hits           4315    35539    +31224     
- Misses         8236    28512    +20276     
- Partials        340     2395     +2055     
Components Coverage Δ
spanner-templates 72.83% <ø> (∅)
spanner-import-export 68.64% <ø> (∅)
spanner-live-forward-migration 80.93% <ø> (∅)
spanner-live-reverse-replication 77.08% <ø> (∅)
spanner-bulk-migration 91.10% <ø> (∅)
gcs-spanner-dv 85.74% <ø> (∅)
Files with missing lines Coverage Δ
...gle/cloud/teleport/bigtable/BigtableToParquet.java 33.96% <0.00%> (ø)

... and 891 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@brandtnewton brandtnewton marked this pull request as ready for review May 11, 2026 20:43
@brandtnewton brandtnewton requested a review from a team as a code owner May 11, 2026 20:43
@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the Bigtable to Parquet template by providing users with the flexibility to configure read row timeouts. By exposing these settings as template parameters, users can now adjust the attempt and operation timeouts to better suit their specific workload requirements, improving the robustness of data migration processes.

Highlights

  • Configuration Updates: Added new configuration options for Bigtable read row timeouts to the BigtableToParquet template.
  • Timeout Customization: Introduced 'Read rows attempt timeout' and 'Read rows operation timeout' parameters, both defaulting to 5 minutes.
  • Implementation: Updated the Bigtable source configuration to utilize the new timeout settings via DurationUtils.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces configurable timeouts for Bigtable read operations in the BigtableToParquet template by adding 'readRowsAttemptTimeout' and 'readRowsOperationTimeout' to the pipeline options. Feedback suggests using 'ValueProvider' for these new options to maintain consistency with the existing codebase, as well as adding '@SuppressWarnings("unused")' to setters and marking the template parameters as optional.

Comment thread v1/src/main/java/com/google/cloud/teleport/bigtable/BigtableToParquet.java Outdated
Copy link
Copy Markdown
Contributor

@georgecma georgecma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay.

@georgecma georgecma added the improvement Making existing code better label May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Making existing code better size/S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants