Sync Repos: Release 170.82.0 #154
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Description This pull request introduces full support for the JSON_ARRAYAGG function in ScriptDom by extending the T-SQL grammar and code generation, along with updated test cases to validate the new functionality. Json_ArrayAgg with options NULL ON NULL/ABSENT ON NULL and RETURNING JSON are supported. Added relevant test for all options. ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [x] Code changes are accompanied by appropriate unit tests - [x] Identified and included SMEs needed to review code changes - [x] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [x] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Additional Information - **`TSql170.g`**: Added a new grammar rule (`jsonArrayAggBuiltInFunctionCall`) and integrated it into the built-in function call productions. - **`SqlScriptGeneratorVisitor.FunctionCall.cs`**: Implemented code generation logic for JSON_ARRAYAGG. - **`CodeGenerationSupporter.cs`**: Defined a new constant for JSON_ARRAYAGG. - **Test updates**: Expanded test scripts and adjusted expected error counts in `JsonFunctionTests170.sql` and `Only170SyntaxTests.cs` to cover JSON_ARRAYAGG scenarios. Related work items: #4519570
# Pull Request Template for ScriptDom ## Description Adding missing JSON index event to the event type enum. Before submitting your pull request, please ensure you have completed the following: ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers Adding missing event CREATE_JSON_INDEX ---- #### AI description (iteration 1) #### PR Classification This PR introduces a new feature by adding a missing event type to the codebase. #### PR Summary The pull request adds the `CreateJsonIndex` event to the `EventNotificationEventType.cs` file with an enum value of 343, ensuring support for the CREATE_JSON_INDEX event. - `SqlScriptDom/Parser/TSql/EventNotificationEventType.cs`: Added the `CreateJsonIndex` enum member along with its documentation. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
Please provide a detailed description, include the link to the design specification or SQL feature document for the new TSQL syntaxes. Make sure to add links to the Github or DevDiv issue Regexp_like function works as a boolean expression similar to LIKE, MATCH starting from compat level 170. Before submitting your pull request, please ensure you have completed the following: - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [x] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code - [x] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file Please provide any additional information that might be helpful for the reviewers Adding regexp_like syntax ---- This pull request implements a new feature by adding support for the REGEXP_LIKE predicate syntax in SQL. The pull request introduces complete support for the REGEXP_LIKE predicate by updating the script generator, grammar, AST, and tests. - `SqlScriptGeneratorVisitor.RegexpLikePredicate.cs`: Added a new visitor method for generating the REGEXP_LIKE SQL syntax. - `TSql170.g`: Extended the T-SQL grammar to include a new rule for REGEXP_LIKE with an optional flags parameter. - `Ast.xml`: Defined a new `RegexpLikePredicate` AST node with members for text, pattern, and optional flags. - `CodeGenerationSupporter.cs`: Added a new constant for REGEXP_LIKE token generation. - Test files in `/Test/SqlDom/Baselines170/RegexpLikeTests170.sql`, `/Test/SqlDom/TestScripts/RegexpLikeTests170.sql`, and updated entry in `Only170SyntaxTests.cs`: Provided extensive test cases validating various REGEXP_LIKE scenarios. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4005966
In this PR we modify the Fabric DW Grammar to support Nested Common Table Expressions with the following syntax:
```
WITH <NESTED_CTE_NAME_LEVEL1> [ (column_name , ...) ] AS
(WITH <NESTED_CTE_NAME_LEVEL2> [ (column_name , ...) ] AS
(
...
WITH <NESTED_CTE_NAME_LEVELn-1> [ ( column_name , ...) ] AS
(
WITH <NESTED_CTE_NAME_LEVELn> [ ( column_name , ...) ] AS
(
Standard_CTE_query_definition
)
<SELECT statement> -- Data source must include NESTED_CTE_NAME_LEVELn
)
<SELECT statement> -- Data source must include NESTED_CTE_NAME_LEVELn-1
...
)
<SELECT statement> -- Data source must include NESTED_CTE_NAME_LEVEL2
)
```
This is derived from public [Nested CTE Documentation](https://learn.microsoft.com/en-us/sql/t-sql/queries/nested-common-table-expression?view=sql-server-ver16).
Work Item: https://dev.azure.com/powerbi/AI/_workitems/edit/1672555/
----
#### AI description (iteration 1)
#### PR Classification
This pull request implements a new feature for supporting nested CTE syntax in Fabric DW.
#### PR Summary
The changes extend the SQL parser grammar and AST to handle nested CTEs with associated XML namespaces while adding tests to verify the new behavior.
- **`SqlScriptDom/Parser/TSql/TSqlFabricDW.g`**: Modified the commonTableExpression rule to include an alternative branch that captures nested CTEs and the corresponding XML namespaces.
- **`SqlScriptDom/Parser/TSql/Ast.xml`**: Introduced a new nullable member `WithCtesAndXmlNamespaces` to support the enhanced syntax.
- **Test files (`OnlyFabricDWSyntaxTests.cs`, new baseline and test scripts)**: Added and referenced nested CTE test cases to validate the implementation for Fabric DW.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
# Pull Request Template for ScriptDom ## Description This PR changes the optional parameter ENABLE_CHUNK_SET_ID of AI_GENERATE_CHUNKS from expression to integer/null only. Corresponding DsmainDev PR: https://msdata.visualstudio.com/Database%20Systems/_git/DsMainDev/pullrequest/1770144 ## Code Change - [X] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [X] Code changes are accompanied by appropriate unit tests - [X] Identified and included SMEs needed to review code changes - [X] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [X] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [X] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers ---- #### AI description (iteration 1) #### PR Classification This PR implements a bug fix by tightening the parsing rules for the ENABLE_CHUNK_SET_ID parameter in AI_GENERATE_CHUNKS and updating related error tests. #### PR Summary The changes restrict the ENABLE_CHUNK_SET_ID parameter to only accept integer constants or NULL literals, and update test expectations accordingly. - `Test/SqlDom/ParserErrorsTests.cs`: Added multiple error tests to validate that invalid values (parameter, column, decimal, string, and function calls) for ENABLE_CHUNK_SET_ID are correctly rejected. - `SqlScriptDom/Parser/TSql/TSql170.g`: Modified the grammar to allow only an integer or NULL literal for ENABLE_CHUNK_SET_ID. - `Test/SqlDom/Baselines170/AiGenerateChunksTests170.sql` and `Test/SqlDom/TestScripts/AiGenerateChunksTests170.sql`: Removed baseline queries using invalid ENABLE_CHUNK_SET_ID values. - `Test/SqlDom/Only170SyntaxTests.cs`: Updated expected error counts for AiGenerateChunks tests to reflect the revised parsing rules. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4566478
# Pull Request Template for ScriptDom ## Description Some of json functions accept json key value and the key value can be with or without single quote. The problem comes when the parser assumes any name and ':' should be parsed as a label. and label are not supported as function parameters. since it's not easy to differentiate between a label and json value, had to change the parser to accept label as function parameter and then parse it as column reference. Before submitting your pull request, please ensure you have completed the following: ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers Fixed a bug when parsing json keyvalue parameter ---- #### AI description (iteration 1) #### PR Classification Bug fix to address the incorrect parsing of JSON keyvalue parameters. #### PR Summary This pull request fixes a parsing issue by updating the T-SQL parser to correctly handle JSON keyvalue expressions—especially when JSON keys are provided as label tokens. The changes improve identifier generation for JSON keys and update tests to verify the behavior. - **`SqlScriptDom/Parser/TSql/TSql160.g` & `SqlScriptDom/Parser/TSql/TSql170.g`**: Extended the JSON keyvalue expression grammar to support label tokens that are converted into identifiers. - **`SqlScriptDom/Parser/TSql/TSql80ParserBaseInternal.cs`**: Introduced the `CreateIdentifierFromLabel` helper method to correctly create identifiers from label tokens. - **Test Files**: Updated changes in multiple test files (e.g., `Test/SqlDom/Baselines170/JsonFunctionTests170.sql`, `Test/SqlDom/TestScripts/JsonFunctionTests170.sql`, and `Test/SqlDom/Only170SyntaxTests.cs`) to confirm the new parsing behavior. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
# Pull Request Template for ScriptDom ## Description Please provide a detailed description, include the link to the design specification or SQL feature document for the new TSQL syntaxes. Make sure to add links to the Github or DevDiv issue Before submitting your pull request, please ensure you have completed the following: ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers Adding release notes for 170.0.96 ---- #### AI description (iteration 1) #### PR Classification This pull request is a documentation update that adds a new release notes file. #### PR Summary The PR introduces a new file detailing the release notes for the release version. - `release-notes/170/170.96.0.md`: Added file containing sections on target platform support, dependency updates, and new features including support for `JSON_ARRAYAGG`, `REGEXP_LIKE`, and 32-bit float vectors. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
# Pull Request Template for ScriptDom ## Description Please provide a detailed description, include the link to the design specification or SQL feature document for the new TSQL syntaxes. Make sure to add links to the Github or DevDiv issue Before submitting your pull request, please ensure you have completed the following: ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers remove release note about vector type change
llali
requested review from
chlafreniere,
dzsquared,
kburtram,
ssreerama and
zijchen
as code owners
ssreerama
approved these changes
Sep 3, 2025
dzsquared
approved these changes
Sep 3, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Sync Repos: Release 170.82.0
Code Changes