fix: update workspace schema exporting endpoint #182
Conversation
…o cover for both .json and .yaml outputs.
WalkthroughThe pull request extends the codebase to support JSON schema files alongside YAML files across CLI generation commands and utility functions. It includes a changeset documenting breaking changes from the Xano Metadata API affecting export-schema endpoints, with fallback parsing logic to handle both JSON and YAML formats. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/utils/src/methods/fetch-and-extract-yaml.ts (1)
6-10: Update the function documentation to reflect dual-format support.The comment on line 8 states "returns the path to the YAML file" but the function now returns either a JSON or YAML schema file path. The function name
fetchAndExtractYamlis also misleading.While the PR objectives mention that renaming will be done in a follow-up patch, the comment should be updated now for accuracy:
/** * Fetches a workspace schema archive from the API, - * extracts it, and returns the path to the YAML file. + * extracts it, and returns the path to the schema file (.json or .yaml). */ export async function fetchAndExtractYaml({
🤖 Fix all issues with AI Agents
In @packages/cli/src/node-config-storage.ts:
- Around line 294-306: The code uses fs.promises.readdir(tempDir, { recursive:
true }) which requires Node >=18.17.0; instead remove the recursive option and
implement a small recursive directory walker: replace the single readdir call
with a helper async function (e.g., walkDir(dir)) that calls
fs.promises.readdir(dir), iterates entries, builds fullPath with join(dir,
entry), calls fs.promises.stat(fullPath), recursively calls walkDir(fullPath)
for directories and for files checks extensions (endsWith('.yaml') ||
endsWith('.json')) then reads the file into result using the same keys you used
before (ensure you preserve how you derive the key from file/fullPath relative
to tempDir); update the call site to await walkDir(tempDir) so behavior matches
the original without requiring the recursive readdir option.
🧹 Nitpick comments (3)
.changeset/dry-files-float.md (1)
6-6: Clarify the changeset description.The wording "fixing a breaking change" is slightly contradictory. This PR is actually restoring compatibility after an external API breaking change, not introducing one. Consider rephrasing to something clearer like:
-fix: **BREAKING CHANGE** fixing a breaking change in the Xano Metadata API for the workspace/{workspace_id}/export-schema endpoint causing the repo and internal docs generator commands to fail +fix: restore compatibility after breaking change in the Xano Metadata API for the workspace/{workspace_id}/export-schema endpoint that caused the repo and internal docs generator commands to failpackages/utils/src/methods/fetch-and-extract-yaml.ts (1)
48-56: Clarify behavior when both .json and .yaml files are present.The current logic uses
find()which returns the first matching file from thereaddir()results. Since filesystem order is not guaranteed, the behavior is unpredictable if both formats exist in the archive. Given that.jsonis the new format and.yamlis legacy, consider explicitly prioritizing.json:🔎 Proposed fix to prioritize .json over .yaml
- const schemaFile = files.find((f) => f.endsWith('.json') || f.endsWith('.yaml')); + const schemaFile = files.find((f) => f.endsWith('.json')) || files.find((f) => f.endsWith('.yaml'));This ensures
.jsonis always selected when present, with.yamlas a fallback.packages/cli/src/commands/generate/implementation/internal-docs.ts (1)
99-115: Parsing logic is sound; consider minor refinements.The multi-format parsing implementation correctly handles JSON and YAML with a sensible fallback strategy. The code uses js-yaml v4.1.0, where
loadis safe by default, so YAML parsing is secure. Error handling and extension-based branching are appropriate.Optional refinements:
Variable naming:
jsonDatais misleading since it holds YAML-parsed content as well. Consider renaming toschemaDataorparsedDatafor clarity.Case-insensitive extension checking: The current
.endsWith()checks are case-sensitive. For robustness, consider normalizing the filename to lowercase before checking extensions (e.g.,inputFile.toLowerCase().endsWith('.json')).
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
.changeset/dry-files-float.mdpackages/cli/src/commands/generate/implementation/internal-docs.tspackages/cli/src/commands/generate/implementation/repo.tspackages/cli/src/commands/generate/index.tspackages/cli/src/node-config-storage.tspackages/utils/src/methods/fetch-and-extract-yaml.ts
🧰 Additional context used
📓 Path-based instructions (3)
packages/*/src/**/*.ts?(x)
📄 CodeRabbit inference engine (AGENTS.md)
packages/*/src/**/*.ts?(x): Use TypeScript for all source code in the monorepo
Configure and use Eslint and Prettier for linting and formatting with default configs
Files:
packages/utils/src/methods/fetch-and-extract-yaml.tspackages/cli/src/commands/generate/implementation/repo.tspackages/cli/src/node-config-storage.tspackages/cli/src/commands/generate/index.tspackages/cli/src/commands/generate/implementation/internal-docs.ts
**/*.ts?(x)
📄 CodeRabbit inference engine (AGENTS.md)
Use js-yaml for YAML processing
Files:
packages/utils/src/methods/fetch-and-extract-yaml.tspackages/cli/src/commands/generate/implementation/repo.tspackages/cli/src/node-config-storage.tspackages/cli/src/commands/generate/index.tspackages/cli/src/commands/generate/implementation/internal-docs.ts
packages/cli/src/**/*.ts?(x)
📄 CodeRabbit inference engine (AGENTS.md)
Use Commander.js for CLI command registration and parsing
Files:
packages/cli/src/commands/generate/implementation/repo.tspackages/cli/src/node-config-storage.tspackages/cli/src/commands/generate/index.tspackages/cli/src/commands/generate/implementation/internal-docs.ts
🧠 Learnings (1)
📚 Learning: 2025-11-28T05:59:33.267Z
Learnt from: CR
Repo: calycode/xano-tools PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-28T05:59:33.267Z
Learning: Applies to **/*.ts?(x) : Use js-yaml for YAML processing
Applied to files:
packages/utils/src/methods/fetch-and-extract-yaml.tspackages/cli/src/node-config-storage.tspackages/cli/src/commands/generate/implementation/internal-docs.ts
🧬 Code graph analysis (2)
packages/utils/src/methods/fetch-and-extract-yaml.ts (2)
packages/utils/src/methods/path.ts (1)
joinPath(2-7)packages/utils/src/index.ts (1)
joinPath(5-5)
packages/cli/src/commands/generate/implementation/repo.ts (1)
packages/core/src/features/oas/generate/methods/cleanup-response-schemas.ts (3)
method(119-200)path(118-201)normalizeToJsonSchema(34-105)
🔍 Remote MCP DeepWiki
Summary of Relevant Context for PR #182 Review
Repository Architecture & Schema Handling
The calycode/xano-tools project is a monorepo with CLI and Core packages. The workspace schema file has historically been YAML-only, with commands like generate-repo and generate-internal-docs explicitly parsing input as YAML using the js-yaml library.
Affected Commands and Their Dependencies
generate-repoandgenerate-internal-docs: Both commands process workspace schemas from either local files (via--input) or by fetching from the Xano Metadata API (via--fetch).- Both commands are registered in
packages/cli/src/commands/generate/index.ts(updated in this PR with expanded option descriptions).
Critical Utility: fetchAndExtractYaml
The fetchAndExtractYaml utility in packages/utils/src/methods/fetch-and-extract-yaml.ts (modified in this PR) is central to the --fetch workflow:
- Fetches workspace schema from Xano Metadata API as a
.tar.gzarchive - Relies on
core.storage.tarExtract()to decompress and extract files - Previously searched only for
.yamlfiles
Storage Layer Change Impact
The tarExtract function in packages/cli/src/node-config-storage.ts (also modified in this PR) was the bottleneck: it previously extracted only .yaml files from the tarball. The PR updates it to recursively traverse and collect both .json and .yaml files, enabling support for Xano's new endpoint behavior that returns workspace.json instead of workspace.yaml.
Scope of Change
This PR addresses the cascading impact of Xano's Metadata API change (mentioned in the changeset as a BREAKING CHANGE):
- Core parsing logic in
repo.tsandinternal-docs.tsnow supports JSON-first fallback to YAML - File discovery and extraction updated at the storage layer
- User-facing CLI descriptions clarified to mention dual format support
🔇 Additional comments (4)
packages/cli/src/commands/generate/index.ts (1)
71-71: LGTM! Clear and accurate option descriptions.The updated descriptions accurately reflect the dual-format support and appropriately mark
.yamlas the legacy format. The consistency between both commands (docs and repo) is good practice.Also applies to: 137-140
packages/cli/src/commands/generate/implementation/repo.ts (1)
110-126: Well-designed multi-format parsing with fallback logic.The parsing implementation is solid:
- Explicit handling for
.json,.yaml, and.ymlextensions- Smart fallback for unknown extensions (tries JSON first, then YAML)
- Proper error handling with descriptive messages
This defensive approach ensures robust parsing across various input scenarios.
packages/cli/src/commands/generate/implementation/internal-docs.ts (2)
93-93: LGTM – Clear dual-format validation message.The updated error message accurately reflects support for both JSON and YAML schema files.
96-96: LGTM – Format-agnostic log message.Using "schema file" instead of "YAML file" appropriately reflects the dual-format support.
| try { | ||
| if (inputFile.endsWith('.json')) { | ||
| jsonData = JSON.parse(fileContents); | ||
| } else if (inputFile.endsWith('.yaml') || inputFile.endsWith('.yml')) { |
There was a problem hiding this comment.
Inconsistency: .yml extension not supported in fetch path.
This code correctly handles .yml extension (line 114), but packages/utils/src/methods/fetch-and-extract-yaml.ts at line 50 only checks for .json and .yaml extensions. This means:
- Manual input with
.ymlworks ✓ - Fetched schemas with
.ymlfail ✗
For consistency, the fetch utility should also check for .yml:
- const schemaFile = files.find((f) => f.endsWith('.json') || f.endsWith('.yaml'));
+ const schemaFile = files.find((f) => f.endsWith('.json')) || files.find((f) => f.endsWith('.yaml') || f.endsWith('.yml'));This can be combined with the priority fix suggested earlier in fetch-and-extract-yaml.ts.
Committable suggestion skipped: line range outside the PR's diff.
| const entries = await fs.promises.readdir(tempDir, { recursive: true }); | ||
|
|
||
| for (const file of entries) { | ||
| // Check for both extensions | ||
| if (file.endsWith('.yaml') || file.endsWith('.json')) { | ||
| const fullPath = join(tempDir, file); | ||
| // Ensure we are reading a file, not a directory that happens to end in .json | ||
| const stat = await fs.promises.stat(fullPath); | ||
| if (stat.isFile()) { | ||
| result[file] = await fs.promises.readFile(fullPath); | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
fd -t f 'package.json' --exec jq -r '.engines.node // "not specified"' {} \;Repository: calycode/xano-tools
Length of output: 182
🏁 Script executed:
# Check the actual code at the specified lines in the file
cat -n packages/cli/src/node-config-storage.ts | sed -n '294,306p'Repository: calycode/xano-tools
Length of output: 741
🏁 Script executed:
# Get more context around the tarExtract method to understand the full implementation
cat -n packages/cli/src/node-config-storage.ts | sed -n '280,315p'Repository: calycode/xano-tools
Length of output: 1472
🏁 Script executed:
cat packages/cli/package.json | jq '.engines.node // "not specified"'Repository: calycode/xano-tools
Length of output: 73
🏁 Script executed:
# Check root package.json if it exists
if [ -f package.json ]; then cat package.json | jq '.engines.node // "not specified"'; fiRepository: calycode/xano-tools
Length of output: 78
🌐 Web query:
Node.js fs.promises.readdir recursive option version requirement
💡 Result:
The fs.promises.readdir({ recursive: true }) option was added in Node.js v20.1.0 and backported to v18.17.0 — so you need Node >= 18.17.0 (or any v20.1.0+ release). [1][2]
Sources: Node.js API docs. [1] [2]
Update minimum Node.js version or refactor readdir usage.
The recursive: true option in fs.promises.readdir() requires Node.js >= 18.17.0, but the project's package.json specifies engines.node: ">=18.0.0". This creates a runtime incompatibility for users on Node.js 18.0.0–18.16.x.
Either:
- Update
package.jsonto specify">=18.17.0"(or">=20.1.0") - Refactor to collect files without the
recursiveoption
🤖 Prompt for AI Agents
In @packages/cli/src/node-config-storage.ts around lines 294 - 306, The code
uses fs.promises.readdir(tempDir, { recursive: true }) which requires Node
>=18.17.0; instead remove the recursive option and implement a small recursive
directory walker: replace the single readdir call with a helper async function
(e.g., walkDir(dir)) that calls fs.promises.readdir(dir), iterates entries,
builds fullPath with join(dir, entry), calls fs.promises.stat(fullPath),
recursively calls walkDir(fullPath) for directories and for files checks
extensions (endsWith('.yaml') || endsWith('.json')) then reads the file into
result using the same keys you used before (ensure you preserve how you derive
the key from file/fullPath relative to tempDir); update the call site to await
walkDir(tempDir) so behavior matches the original without requiring the
recursive readdir option.
1 participant
output file handler to cover for both .json and .yaml outputs.
Context: Xano has updated their
workspace/{workspace_id}/export-schemaendpoint to return a .tar.gz archive with a singleworkspace.jsonfile instead of the previously deliveredworkspace.yaml.While I welcome this change as this now enables more lightweight parsing of the file in browser environments as well, this change did come as a surprise and only got noticed due to failing CI/CD on one of my instances and when I tried to use my CLI to pull in the latest state of a client instance. This is very much unfortunate.
This PR just wants to get the fix out as quick as possible, the function and method naming updates (to also express this double format nature) will follow in a patch update later this month.
Summary by CodeRabbit
Release Notes
New Features
Bug Fixes
Chores
✏️ Tip: You can customize this high-level summary in your review settings.