Sync Repos: Release 170.82.0 #151
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Pull Request Template for ScriptDom ## Description This PR adds tests for regex intrinsic functions. Functional Spec - https://microsoft.sharepoint.com/:w:/t/Ad-Astra/EZAk15L-vkJHqsaCybly0QIB0eBUdv02BBaHUAl48aY_Pw?e=NPRj0w ## Code Change - [x] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [x] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [x] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [x] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information None Adding regex functions tests Related work items: #4005966
# Pull Request Template for ScriptDom
## Description
# Add syntax and script generation support for AI_GENERATE_CHUNKS table-valued function
## Description
This PR introduces parser and script generator support for the new built-in table-valued function (TVF) `AI_GENERATE_CHUNKS`.
`AI_GENERATE_CHUNKS` enables chunking of input text based on a specified strategy. It accepts required and optional parameters and supports aliasing for cross apply scenarios.
### Syntax supported
```sql
SELECT * FROM AI_GENERATE_CHUNKS (
source = 'some text',
chunk_type = fixed,
chunk_size = 5,
overlap = 2,
enable_chunk_set_id = 1
);
```
### Parameters
| Parameter | Required | Type | Description |
|------------------------|----------|-------------------|--------------------------------------------------------------------|
| `source` | Yes | `ScalarExpression`| Input text to be chunked |
| `chunk_type` | Yes | `Identifier` | Type of chunking strategy. Currently only `fixed` is supported |
| `chunk_size` | Yes | `ScalarExpression`| Number of tokens per chunk |
| `overlap` | No | `ScalarExpression`| Optional token overlap between adjacent chunks |
| `enable_chunk_set_id` | No | `ScalarExpression`| Optional flag. If set, adds a `chunk_set_id` column to the result |
### Summary of Changes
- Added support for the `ai_generate_chunks` table-valued function to the SQL parser.
- Introduced new AST class `AiGenerateChunksTableReference` with members:
- `Source`
- `ChunkType`
- `ChunkSize`
- `Overlap`
- `EnableChunkSetId`
- Added grammar rule for parsing `ai_generate_chunks`.
- Implemented corresponding visitor logic for script generation (`SqlScriptGeneratorVisitor`).
- Added unit tests for:
- Positive cases covering all combinations of optional parameters.
- Negative cases for missing/invalid parameter order, unexpected tokens, or malformed expressions.
### Example Use Cases
This function is designed to support downstream AI workflows where long documents need to be split into manageable token chunks for embedding generation or other ML tasks.
## Code Change
- [X] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed
- [X] Code changes are accompanied by appropriate unit tests
- [X] Identified and included SMEs needed to review code changes
- [X] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code
## Testing
- [X] Follow the [steps](https://msdata.vi...
…rning JSON option for json_object, json_array # Pull Request Template for ScriptDom ## Description adds support to scriptdom for new functions and options in SQLServer. Specifically support for - Json_objectagg with options NULL ON NULL/ABSENT ON NULL and RETURNING JSON. - Support for RETURNING JSON option for json_object, json_array. Added relevant test for all options. ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [X] Code changes are accompanied by appropriate unit tests - [X] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [X] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature Added Test.
# Pull Request Template for ScriptDom
## Description
This PR adds support for the new `AI_GENERATE_EMBEDDINGS` T-SQL built-in function syntax:
```sql
AI_GENERATE_EMBEDDINGS('input text' USE MODEL model_name [PARAMETERS (...optional JSON...)])
```
Input expression: required string or expression representing the text to embed.
USE MODEL model_name: required keyword clause specifying the embedding model.
PARAMETERS ( ... ): optional block allowing additional JSON parameters to customize embedding behavior.
### Examples
```sql
-- Basic usage with required input and model
SELECT AI_GENERATE_EMBEDDINGS('Hello world' USE MODEL MyEmbeddingModel);
-- Usage with optional PARAMETERS block
SELECT AI_GENERATE_EMBEDDINGS('Hello world' USE MODEL MyEmbeddingModel PARAMETERS (TRY_CONVERT(JSON, N'{"param":"value"}')));
```
### Features
- Adds `AiGenerateEmbeddingsFunctionCall` AST node.
- Grammar recognizes required `USE MODEL` clause and optional `PARAMETERS` block.
- Script generator visitor outputs syntax matching expected SQL Server formatting, with or without optional PARAMETERS.
- Comprehensive unit tests cover:
- Missing required keywords or values (e.g., missing USE, missing MODEL).
- Misplaced keywords (e.g., PARAMETERS before input).
- Invalid syntax like missing parentheses or incomplete expressions.
- Extra commas or unknown parameters.
- Negative tests aligned with expected SQL46010 and SQL46029 parser errors.
- Supports round-tripping: parsed AST can be pretty-printed back to valid SQL matching the original input.
### Compatibility
- Valid only in SQL Server with compatibility level 170 or higher.
## Code Change
- [X] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed
- [X] Code changes are accompanied by appropriate unit tests
- [X] Identified and included SMEs needed to review code changes
- [X] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code
## Testing
- [X] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature
## Documentation
- [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file
----
#### AI description (iteration 1)
#### PR Classification
New feature implementation to add syntax support for the AI_GENERATE_EMBEDDINGS function in ScriptDom.
#### PR Summary
This PR introduces a new SQL function, AI_GENERATE_EMBEDDINGS, which allows users to send input text to external AI models and retrieve an em...
…odel support
# Pull Request Template for ScriptDom
## Description
Introduces syntax support for Create, Alter and Drop operation for External Model DDL
CREATE
CREATE EXTERNAL MODEL <model_name>
AUTHORIZATION owner
WITH
( LOCATION = 'URL or FILE PATH',
API_FORMAT = ‘OpenAI, Azure OpenAI, etc’,
MODEL_TYPE = <EMBEDDINGS,CHAT, etc>,
MODEL = ‘text-embedding-ada-002,etc’,
CREDENTIAL = <credential_name>,
PARAMETERS = ‘{ “valid”:”JSON”}’
}
| **Column Name** | **Data Type** | **Notes** |
|-----------------------|---------------------|-----------|
| `model_name` | `SYSNAME` | Name of the external model object |
| `owner_name` | `SYSNAME` | Database principal that owns the external model object. Defaults to connected user if not specified. |
| `location` | `NVARCHAR(4000)` | Location of the endpoint. Can be a URL (`https://...`) or file path (`mountpoint:\\...`). |
| `api_format` | `NVARCHAR(100)` | Format of the API the endpoint accepts/produces. P0: Azure OpenAI, OpenAI, Ollama; P1: Local, Nvidia Triton, Databricks; P2: TBD |
| `model_type` | `SYSNAME` | Model type for the data plane. P0: EMBEDDINGS; Post GA: CHAT, MODERATE, TRANSLATE, SPEECH. Only supported types are accepted. |
| `model` | `NVARCHAR(100)` | Name of the embedding model used by the endpoint. |
| `local_runtime_path` | `NVARCHAR(4000)` | Path to local model runtime (e.g., ONNX runtime). Required for local models. |
| `credential` | `SYSNAME` | Name of the database scoped credential. Not used for ONNX Runtime. |
| `parameters` | `JSON` | Default JSON parameters to be appended to the embeddings payload. |
## Code Change
- [X] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed
- [X] Code changes are accompanied by appropriate unit tests
- [X] Identified and included SMEs needed to review code changes
- [X] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code
## Testing
- [X] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature
## Documentation
- [X] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file
## Additional Information
Please provide any additional information that might be helpful for the reviewers
ScriptDom channges
----
#### AI description (iteration 1)
#### PR Classific...
# Pull Request Template for ScriptDom ## Description Added parser and script generator support for VECTOR_SEARCH Before submitting your pull request, please ensure you have completed the following: ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers Add ScriptDom support for VECTOR_SEARCH function ---- #### AI description (iteration 1) #### PR Classification This pull request introduces a new feature by adding native ScriptDom support for the VECTOR_SEARCH function. #### PR Summary The changes implement parsing, AST definition, and script generation for VECTOR_SEARCH, along with a comprehensive suite of tests to validate error handling and syntax compliance. - **`SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.VectorSearchTableReference.cs`**: New file added to generate script output for the VECTOR_SEARCH function. - **`SqlScriptDom/Parser/TSql/TSql170.g` & `SqlDom/Parser/TSql/Ast.xml`**: Extended grammar rules and AST definitions to support VECTOR_SEARCH. - **`SqlScriptDom/Parser/TSql/CodeGenerationSupporter.cs`**: Added new constants (e.g., COSINE, DOT, EUCLIDEAN, METRIC, TOP_N, VECTOR_SEARCH) required for vector search processing. - **`Test/SqlDom/ParserErrorsTests.cs` and related vector test scripts**: Introduced extensive tests covering syntax errors and parameter validations for VECTOR_SEARCH. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4440435
## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers Added EM Permission support ---- #### AI description (iteration 1) #### PR Classification This PR implements a new feature by adding support for External Model permissions in ScriptDOM, enabling the parsing and generation of new permission-related T-SQL statements. #### PR Summary The pull request enhances ScriptDOM to correctly handle External Model permissions—covering GRANT, DENY, REVOKE, and ALTER AUTHORIZATION operations—and integrates comprehensive tests and token generation support to improve IntelliSense and syntax-highlighting as outlined in the linked work items. - **`Test/SqlDom/ExternalModelPermissionUnitTests.cs`**: Adds unit tests for External Model permission statements, including backward compatibility checks. - **`Test/SqlDom/Baselines170/SecurityStatementExternalModelTests170.sql` & `Test/SqlDom/TestScripts/SecurityStatementExternalModelTests170.sql`**: Introduce baseline and test scripts with various External Model permission scenarios. - **`SqlScriptDom/Parser/TSql/TSql170ParserBaseInternal.cs` & `SqlScriptDom/Parser/TSql/SecurityObjectKind.cs`**: Update parsing logic and the security object enumeration to recognize and handle External Model keywords. - **`SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.SecurityTargetObject.cs`**: Enhances token generation to support the External Model permission syntax. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4088625, #4088633
# Pull Request Template for ScriptDom ## Description Please provide a detailed description, include the link to the design specification or SQL feature document for the new TSQL syntaxes. Make sure to add links to the Github or DevDiv issue Before submitting your pull request, please ensure you have completed the following: ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers Adding release notes for 170.82.0
llali
requested review from
chlafreniere,
dzsquared,
kburtram,
ssreerama and
zijchen
as code owners
ssreerama
approved these changes
Aug 7, 2025
dzsquared
approved these changes
Aug 7, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Please provide a detailed description. Be as descriptive as possible - include information about what is being changed,
why it's being changed, and any links to relevant issues. If this is closing an existing issue use one of the issue linking keywords to link the issue to this PR and have it automatically close when completed.
In addition, go through the checklist below and check each item as you validate it is either handled or not applicable to this change.
Code Changes