Skip to content

feat(extension): new extension for generic storage filesystem query#2003

Open
brunolnetto wants to merge 3 commits into
duckdb:mainfrom
brunolnetto:main
Open

feat(extension): new extension for generic storage filesystem query#2003
brunolnetto wants to merge 3 commits into
duckdb:mainfrom
brunolnetto:main

Conversation

@brunolnetto

Copy link
Copy Markdown

This pull request adds a new extension metadata file, description.yml, for the cloudfs extension. The file provides a comprehensive overview of the extension's capabilities, supported providers, authentication methods, and usage examples.

Key additions in the new extension metadata:

Extension Definition and Capabilities

  • Introduces the cloudfs extension, which enables querying multiple cloud storage providers (SharePoint, OneDrive, Google Drive, Dropbox, SFTP, and VPS via cloudfs-agent) using custom URL schemes (spfs://, odfs://, gdfs://, dbxfs://, sftp://, and vfs://) with the same features as s3://, such as support for Parquet, CSV, JSON, Delta Lake, Iceberg, glob patterns, and COPY TO operations.
  • Specifies extension metadata including version, language (C++), build system (cmake), license (MIT), maintainers, and required toolchains.

Documentation and Usage

  • Provides example usage for authenticating with SharePoint, reading Parquet files, and interacting with a VPS using cloudfs-agent.
  • Details the supported URL schemes, providers, and authentication methods in a clear table, and explains compatibility with various file formats and features.
  • Describes deployment

Copilot AI review requested due to automatic review settings June 3, 2026 16:50

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds an extension manifest/description file for the cloudfs DuckDB extension, including repository pinning and end-user documentation.

Changes:

  • Introduces description.yml with extension metadata (name, version, toolchains, maintainers).
  • Adds embedded hello_world and extended documentation for supported schemes/providers and deployment hints.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread description.yml Outdated
Comment thread description.yml Outdated
Comment thread description.yml Outdated
brunolnetto added a commit to trouchet/cloudfs that referenced this pull request Jun 3, 2026
Fix 3 issues from GitHub Copilot review:

1. Change licence → license (American spelling, schema compliance)
2. Convert requires_toolchains from semicolon string to YAML list
   - Better validation and cleaner diffs
   - Easier to parse and extend
3. Fix grammar: a ~8 MB → an ~8 MB (vowel sound rule)

Addresses: github.com/duckdb/community-extensions/pull/2003 review
@carlopi

carlopi commented Jun 14, 2026

Copy link
Copy Markdown
Member

Hi @brunolnetto, I think the descriptor yaml needs to be in a folder with the extension name, could you move that so it's at ./extensions/cloudfs/description.yml ? Thanks!

@brunolnetto

Copy link
Copy Markdown
Author

Hi @brunolnetto, I think the descriptor yaml needs to be in a folder with the extension name, could you move that so it's at ./extensions/cloudfs/description.yml ? Thanks!

Addressed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants