The current fallback strategy for oracle db uses complete table iteration, which might be really slow for large tables. To avoid such delays in chunking an extent strategy can be implemented. A draft code for this strategy is already added and commented out, which can be used as a reference for implementation.
|
// TODO: Add support for oracle extents based chunking strategy |
|
/* |
|
splitViaExtents manually chunks a table by using extents of the table in OracleDB |
|
func (o *Oracle) splitViaExtents(ctx context.Context, stream types.StreamInterface, blocksPerChunk int64) (*types.Set[types.Chunk], error) { |
|
chunks := types.NewSet[types.Chunk]() |
|
|
This issue was generated by todo-issue based on a TODO comment in 8804519. It's been assigned to @vaibhav-datazip because they committed the code.
The current fallback strategy for oracle db uses complete table iteration, which might be really slow for large tables. To avoid such delays in chunking an extent strategy can be implemented. A draft code for this strategy is already added and commented out, which can be used as a reference for implementation.
olake/drivers/oracle/internal/backfill.go
Lines 212 to 217 in 8804519
This issue was generated by todo-issue based on a
TODOcomment in 8804519. It's been assigned to @vaibhav-datazip because they committed the code.