Merged PR 6131: SYNC: Github main to ADO main

Sync GH to ADO

----
#### AI description  (iteration 1)
#### PR Classification
This PR syncs updates from GitHub main into the ADO main branch, incorporating extensive test additions, pipeline enhancements, and core code improvements.

#### PR Summary
The PR merges comprehensive changes including new tests for cursor/connection and global settings, updated build and PR-validation pipeline configurations for cross-platform wheel builds, and enhancements to the connection and cursor modules (e.g. context manager, execute methods, output converters).
- **`tests/`**: Added and updated numerous tests in files such as `test_003_connection.py` and `test_001_globals.py` to cover execute methods, batch execution, timeout handling, encoding/decoding APIs, exception attributes, and thread‐safety.
- **`eng/pipelines/build-whl-pipeline.yml` & `pr-validation-pipeline.yml`**: Revised YAML pipeline definitions to support manylinux, musllinux, Alpine (both x86_64 and ARM64), and improved artifact collection and test reporting.
- **`mssql_python/cursor.py`**: Enhanced cursor functionality with refined context manager support, fetch methods, scrolling, and error handling.
- **`mssql_python/connection.py`**: Updated connection API including new execute() convenience methods, output converter management (add/get/remove/clear), setencoding/getencoding and setdecoding/getdecoding methods, and timeout support.
- **Removed file**: Deleted the obsolete `/mssql_python/testing_ddbc_bindings.py` as its functionality has been integrated into the core modules.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Related work items: #38881

Author: bewithgaurav

.coveragerc

@@ -1,6 +1,8 @@
 [run]
 omit =
-    mssql_python/testing_ddbc_bindings.py
+    main.py
+    setup.py
+    bcp_options.py
     tests/*
 
 [report]

.github/workflows/pr-code-coverage.yml

@@ -57,9 +57,9 @@ jobs:
               // Extract the summary content
               const summaryContent = summaryMatch[1];
 
-              // Remove all HTML comments including the template placeholder
+              // Remove all HTML comments including unclosed ones (template placeholders)
               const contentWithoutComments = 
-                summaryContent.replace(/<!--[\s\S]*?-->/g, '');
+                summaryContent.replace(/<!--[\s\S]*?(?:-->|$)/g, '');
 
               // Remove whitespace and check if there's actual text content
               const trimmedContent = contentWithoutComments.trim();
@@ -94,24 +94,35 @@ jobs:
               labelToAdd = 'pr-size: large';
             }
 
-            // Remove existing size labels if any
+            // Get existing labels
             const existingLabels = pr.labels.map(l => l.name);
             const sizeLabels = ['pr-size: small', 'pr-size: medium', 'pr-size: large'];
-            for (const label of existingLabels) {
-              if (sizeLabels.includes(label)) {
+            
+            // Find current size label (if any)
+            const currentSizeLabel = existingLabels.find(label => sizeLabels.includes(label));
+            
+            // Only make changes if the label needs to be updated
+            if (currentSizeLabel !== labelToAdd) {
+              console.log(`Current size label: ${currentSizeLabel || 'none'}`);
+              console.log(`Required size label: ${labelToAdd} (Total changes: ${totalChanges})`);
+              
+              // Remove existing size label if different from required
+              if (currentSizeLabel) {
+                console.log(`Removing outdated label: ${currentSizeLabel}`);
                 await github.rest.issues.removeLabel({
                   ...context.repo,
                   issue_number: pr.number,
-                  name: label,
+                  name: currentSizeLabel,
                 });
               }
-            }
 
-            // Add new size label
-            await github.rest.issues.addLabels({
-              ...context.repo,
-              issue_number: pr.number,
-              labels: [labelToAdd],
-            });
-
-            console.log(`Added label: ${labelToAdd} (Total changes: ${totalChanges})`);
+              // Add new size label
+              console.log(`Adding new label: ${labelToAdd}`);
+              await github.rest.issues.addLabels({
+                ...context.repo,
+                issue_number: pr.number,
+                labels: [labelToAdd],
+              });
+            } else {
+              console.log(`Label already correct: ${labelToAdd} (Total changes: ${totalChanges}) - no changes needed`);
+            }

.github/workflows/pr-format-check.yml

@@ -1,6 +1,3 @@
-# Ignore all files in the pybind/build directory
-mssql_python/pybind/build/
-
 # Ignore pycache files and folders
 __pycache__/
 **/__pycache__/
@@ -23,6 +20,7 @@ test-*.xml
 
 # Ignore the build & mssql_python.egg-info directories
 build/
+**/build/
 mssql_python.egg-info/
 
 # Python bytecode
@@ -46,4 +44,19 @@ build/
 *.swp
 
 # .DS_Store files
-.DS_Store
+.DS_Store
+
+# wheel files
+*.whl
+*.tar.gz
+*.zip
+
+# Dockerfiles and images (root only)
+/Dockerfile*
+/docker-compose.yml
+/docker-compose.override.yml
+/docker-compose.*.yml
+
+# Virtual environments
+*venv*/
+**/*venv*/

.gitignore

@@ -1,26 +1,55 @@
 # mssql-python
 
-This is a new Python driver for Microsoft SQL Server currently in Alpha phase.
+mssql-python is a new first-party SQL Server driver for Python that has all of the benefits of a fresh start while preserving a familiar experience for developers.
+
+## What makes mssql-python different?
+
+### Powered by DDBC – Direct Database Connectivity
+
+Most Python SQL Server drivers, including pyodbc, route calls through the Driver Manager, which has slightly different implementations across Windows, macOS, and Linux. This results in inconsistent behavior and capabilities across platforms. Additionally, the Driver Manager must be installed separately, creating friction for both new developers and when deploying applications to servers.
+
+At the heart of the driver is DDBC (Direct Database Connectivity) — a lightweight, high-performance C++ layer that replaces the platform’s Driver Manager.
+
+Key Advantages:
+
+- Provides a consistent, cross-platform backend that handles connections, statements, and memory directly.
+- Interfaces directly with the native SQL Server drivers.
+- Integrates with the same TDS core library that powers the ODBC driver.
+
+### Why is this architecture important?
+
+By simplifying the architecture, DDBC delivers:
+
+- Consistency across platforms
+- Lower function call overhead
+- Zero external dependencies on Windows (`pip install mssql-python` is all you need)
+- Full control over connections, memory, and statement handling
+
+### Built with PyBind11 + Modern C++ for Performance and Safety
+
+To expose the DDBC engine to Python, mssql-python uses PyBind11 – a modern C++ binding library, instead of ctypes. With ctypes, every call between Python and the ODBC driver involved costly type conversions, manual pointer management, resulting in slow and potentially unsafe code.
+
+PyBind11 provides:
+
+- Native-speed execution with automatic type conversions
+- Memory-safe bindings
+- Clean and Pythonic API, while performance-critical logic remains in robust, maintainable C++.
 
 ## Public Preview Release
 
-We are making progress - The Public Preview of our driver is now available! This marks a significant milestone in our development journey. While we saw a few early adopters of our alpha release, we are introducing the following functionalities to support your applications in a more robust and reliable manner.
+We are currently in **Public Preview**.
 
-### What's Included:
+## What's new in v0.12.0
 
-- Everything from previous releases
-- **Azure Active Directory Authentication:** New authentication module supporting Azure AD login options (ActiveDirectoryInteractive, ActiveDirectoryDeviceCode, ActiveDirectoryDefault) for secure and flexible cloud integration.
-- **Batch Execution Performance:** Refactored `executemany` for efficient bulk operations and improved C++ bindings for performance.
-- **Robust Logging System:** Overhauled logging with a singleton manager, sensitive data sanitization, and better exception handling.
-- **Improved Row Representation:** Enhanced output and debugging via updated `Row` object string and representation methods.
+- **Complex Data Type Support:** Added native support for DATETIMEOFFSET and UNIQUEIDENTIFIER data types with full round-trip handling, enabling seamless integration with Python's timezone-aware `datetime` objects and `uuid.UUID` types.
+- **Support for monetary or currency values data types:** Extended MONEY and SMALLMONEY support to `executemany` operations with proper NULL handling and decimal conversion for improved bulk financial data processing.
+- **Improved Database Metadata API:** Added `getinfo()` method with enhanced ODBC metadata retrieval, allowing users to query driver/data source information using ODBC info types.
+- **Data Processing Optimizations:** Removed aggressive datetime parsing to prevent incorrect type conversions and improve data integrity across diverse datetime formats and string data.
 
 For more information, please visit the project link on Github: https://github.com/microsoft/mssql-python
 
-### What's Next:
-
-As we continue to develop and refine the driver, you can expect regular updates that will introduce new features, optimizations, and bug fixes. We encourage you to contribute, provide feedback and report any issues you encounter, as this will help us improve the driver for the final release.
+If you have any feedback, questions or need support please mail us at mssql-python@microsoft.com.
 
-### Stay Tuned:
+## What's Next
 
-We appreciate your interest and support in this project. Stay tuned for more updates and enhancements as we work towards delivering a robust and fully-featured driver in coming months.
-Thank you for being a part of our journey!
+As we continue to develop and refine the driver, you can expect regular updates that will introduce new features, optimizations, and bug fixes. We encourage you to contribute, provide feedback and report any issues you encounter, as this will help us improve the driver ahead of General Availability.

PyPI_Description.md

@@ -17,22 +17,37 @@ pip install mssql-python
 ```
 **MacOS:** mssql-python can be installed with [pip](http://pypi.python.org/pypi/pip)
 ```bash
+# For Mac, OpenSSL is a pre-requisite - skip if already present
 brew install openssl
 pip install mssql-python
 ```
 **Linux:** mssql-python can be installed with [pip](http://pypi.python.org/pypi/pip)
 ```bash
+# For Alpine
+apk add libtool krb5-libs krb5-dev
+
+# For Debian/Ubuntu  
+apt-get install -y libltdl7 libkrb5-3 libgssapi-krb5-2
+
+# For RHEL
+dnf install -y libtool-ltdl krb5-libs
+
+# For SUSE
+zypper install -y libltdl7 libkrb5-3 libgssapi-krb5-2
+
+# For SUSE/openSUSE
+zypper install -y libltdl7
+
 pip install mssql-python
 ```
 
 ## Key Features
 ### Supported Platforms
 
-Windows, MacOS and Linux (manylinux2014 - Debian, Ubuntu & RHEL)
+Windows, MacOS and Linux (manylinux - Debian, Ubuntu, RHEL, SUSE (x64 only) & musllinux - Alpine)
 
 > **Note:**
-> Support for additional Linux OSs (Alpine, SUSE Linux) will come soon
->
+> SUSE Linux ARM64 is not supported by Microsoft ODBC Driver. Use x64 architecture for SUSE deployments.
  
 ### DBAPI v2.0 Compliance
 
@@ -62,10 +77,7 @@ EntraID authentication is now fully supported on MacOS and Linux but with certai
 | ActiveDirectoryDeviceCode | ✅ Yes | ✅ Yes | Device code flow for authentication; suitable for environments without browser access |
 | ActiveDirectoryDefault | ✅ Yes | ✅ Yes | Uses default authentication method based on environment and configuration |
 
-**NOTE**: 
- - **Access Token**: the connection string **must not** contain `UID`, `PWD`, `Authentication`, or `Trusted_Connection` keywords.
- - **Device Code**: make sure to specify a `Connect Timeout` that provides enough time to go through the device code flow authentication process.
- - **Default**: Ensure you're authenticated via az login, or running within a managed identity-enabled environment.
+> For more information on Entra ID please refer this [document](https://github.com/microsoft/mssql-python/wiki/Microsoft-Entra-ID-support)
 
 ### Enhanced Pythonic Features