Implement remote DFS support for LogicalFileListFiltered#170
Draft
Copilot wants to merge 4 commits intoHPCC-35827-LogicalFileListFilteredfrom
Draft
Implement remote DFS support for LogicalFileListFiltered#170Copilot wants to merge 4 commits intoHPCC-35827-LogicalFileListFilteredfrom
Copilot wants to merge 4 commits intoHPCC-35827-LogicalFileListFilteredfrom
Conversation
Introduces new file listing API with user-friendly filter syntax and field selection. LogicalFileList is marked DEPRECATED, users should use new function for efficiency and capability. Features: - Filter syntax: wildcards (owner:jsmith), numeric comparisons (size>1000000), date ranges (modified>2024-01-01), property checks (has:description), file type filtering (is:superfile, is:normal) - Field projection to select specific result fields - Configurable file limits with breach detection - Overflow/underflow protection for numeric filters Implementation: - FileListResultFieldSource for IFieldSource pattern - parseUserFilterSyntax converts user syntax to internal DFUQFilter - Field validation via dfuFieldAttrPathMap - Default fields: name, modified, size, rowcount, cluster, superfile, owner Testing: - 12 regression test cases covering filtering, projection, limits, edge cases Signed-off-by: Jake Smith <jake.smith@lexisnexisrisk.com>
Tighten LogicalFileListFiltered parsing; rename remoteDfs; update regress tests Signed-off-by: Jake Smith <jake.smith@lexisnexisrisk.com>
…se 1 Co-authored-by: jakesmith <902700+jakesmith@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Implement missing remoteDfs side of LogicalFileListFiltered
Implement remote DFS support for LogicalFileListFiltered
Feb 19, 2026
90b89c5 to
c7a19c7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds remote DFS capability to
LogicalFileListFiltered, enabling file listing across remote storage planes. Previously, theremoteDfsparameter threw "not supported yet" exception.Changes
ESDL Interface (
ws_dfs.ecm)DFSListFilteredmethod (version 1.02)Server Handler (
ws_dfsservice.cpp)CWsDfsEx::onDFSListFiltered(): parses user-friendly filters server-side, queries Dali viagetDFAttributesFilteredIterator(), returns LZ-compressed resultsparseUserFilterSyntax(): translatesowner:jsmith, size>1000to internal Dali filter formatvalidateFileField(): field name validation with alias support (superfile→numsubfiles, rowcount→recordcount)Client (
ws_dfsclient.cpp)listFilteredDFSFiles(): discovers remote service viagetRemoteStorage(), handles SSL/auth, sends human-readable parameters, decompresses responsePlugin (
fileservices.cpp)fsLogicalFileListFiltered(): routes to remote service whenremoteDfsspecified, otherwise local Dali query unchangedUsage
Implementation Notes
lookupDFSFilepattern for remote communication, lease management, and retry logicWarning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
downloads.apache.org/usr/bin/curl curl --fail --retry 3 -L REDACTED --create-dirs --output apr-1.7.6.tar.bz2.7325.part -H User-Agent: vcpkg/2025-09-03-4580816534ed8fd9634ac83d46471440edd82dfe (curl) u/13/cc1 --64�� -o conftest.o /snap/bin/as h include/antlr3commontoken.h include/antlr3commontree.h include/antlr3commontreeadaptor.h incl --global ild/vcpkg_downlo. as(dns block)If you need me to access, download, or install something from one of these locations, you can either:
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.