[DEV-4317] add full sync script#205
Open
MarBert wants to merge 7 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a split sync workflow for publishing documentation to the public repository, using incremental syncs when paths are provided and a full sync otherwise.
Changes:
- Adds incremental sync script logic for selected add/remove paths.
- Adds a full repository sync script driven by a remote directory list.
- Updates the GitHub Actions workflow to choose between incremental and full sync and stage commits accordingly.
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
scripts/synch_repo.sh |
Adds incremental copy/remove handling and emits modified files for staging. |
scripts/synch_full_repo.sh |
Adds full docs sync by downloading a directory list, clearing public docs, and copying matching private docs. |
.github/workflows/push_on_public_repo.yml |
Routes workflow execution to incremental or full sync and adjusts staging behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+12
to
+22
| echo "Downloading JSON file from $JSON_URL..." | ||
| JSON_CONTENT=$(curl -s "$JSON_URL") | ||
|
|
||
| echo "Extracting the path list..." | ||
| # Extracts strings between quotes, ignores the "dirNames" key, and removes the quotes | ||
| PATHS=$(echo "$JSON_CONTENT" | grep -o '"[^"]*"' | grep -v '"dirNames"' | tr -d '"') | ||
|
|
||
| echo "Cleaning up the directory $PUBLIC_DOCS_DIR..." | ||
| # Create the directory if it doesn't exist, otherwise empty its contents without deleting the root folder | ||
| mkdir -p "$PUBLIC_DOCS_DIR" | ||
| rm -rf "${PUBLIC_DOCS_DIR:?}/"* |
Comment on lines
+68
to
+69
| chmod +x scripts/sync_full_repo.sh | ||
| ./scripts/sync_full_repo.sh |
| run: | | ||
| chmod +x scripts/synch_repo.sh | ||
| ./scripts/synch_repo.sh | ||
| ./scripts/synch_repo.sh |
Comment on lines
+25
to
+27
| for PATH_NAME in $PATHS; do | ||
| SRC_DIR="$PRIVATE_DOCS_DIR/$PATH_NAME" | ||
| DEST_DIR="$PUBLIC_DOCS_DIR/$PATH_NAME" |
| git add -A -- "$file" | ||
| fi | ||
| done | ||
| if [ -n "${{ inputs.paths_to_add }}" ] || [ -n "${{ inputs.paths_to_remove }}" ]; then |
| done | ||
| if [ -n "${{ inputs.paths_to_add }}" ] || [ -n "${{ inputs.paths_to_remove }}" ]; then | ||
| echo "Running partial commit..." | ||
| echo "${{ steps.sync_step.outputs.modified_files }}" | while IFS= read -r file; do |
…cript-for-full-sync # Conflicts: # .github/workflows/push_on_public_repo.yml # scripts/synch_repo.sh
…cript-for-full-sync # Conflicts: # .github/workflows/push_on_public_repo.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request enhances the repository synchronization workflow by introducing a distinction between incremental and full sync operations, and adds a new script to handle full synchronization. The changes ensure that only the necessary files are synced when specific paths are provided, while a complete sync is performed otherwise. This improves efficiency and clarity in the sync process.
Workflow improvements:
.github/workflows/push_on_public_repo.ymlnow conditionally runs either an incremental sync (usingscripts/synch_repo.sh) when specific paths are provided, or a full sync (using the newscripts/sync_full_repo.sh) when no paths are specified. [1] [2]Script additions:
scripts/synch_full_repo.sh, has been added to handle full repository synchronization. It downloads a list of directory names from a remote JSON file, cleans the public docs directory, and copies the relevant directories from the private to the public repo, with appropriate checks and logging.