Skip to content

Opt sync cds logs #722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AlexCXC wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Opt sync cds logs #722

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2692403
add logs for sync-cds and force-sync-cds about syncing warning
AlexCXC Jan 23, 2025
9a1e473
update log
AlexCXC Jan 23, 2025
2d55053
update
AlexCXC Jan 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions dtable_events/dtable_io/task_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -357,11 +357,14 @@ def add_sync_common_dataset_task(self, context):
"""
return: task_id -> str or None, error_type -> str or None
"""
from dtable_events.dtable_io import dtable_io_logger
from dtable_events.dtable_io.import_sync_common_dataset import sync_common_dataset

dataset_sync_id = context.get('sync_id')
with self.dataset_sync_ids_lock:
if self.is_syncing(dataset_sync_id):
dtable_io_logger.warning('sync_id %s is syncing, current syncing ids: %s',
dataset_sync_id, self.dataset_sync_ids)
return None, 'syncing'
self.add_dataset_sync(dataset_sync_id)

Expand All @@ -378,11 +381,14 @@ def add_force_sync_common_dataset_task(self, context):
"""
return: task_id -> str or None, error_type -> str or None
"""
from dtable_events.dtable_io import dtable_io_logger
from dtable_events.dtable_io.import_sync_common_dataset import force_sync_common_dataset

dataset_id = context.get('dataset_id')
with self.force_sync_dataset_ids_lock:
if self.is_dataset_force_syncing(dataset_id):
dtable_io_logger.warning('dataset %s is syncing, current syncing datasets: %s',
dataset_id, self.force_sync_dataset_ids)
return None, 'syncing'
self.force_sync_dataset_ids.add(dataset_id)

Expand Down