TDS v3 Internals Redesign

[mjaric]

TDS v3 Internals Redesign

Goal: Incrementally rewrite the TDS library internals to deliver a well-layered, well-tested codebase with
user-extensible types, clean protocol layers, correct TLS, and proper error handling. Without breaking the
public API.

Motivation

The current TDS library (v2.x) works but has fundamental architectural problems:

• Monolithic type system - types.ex is 1800+ lines; adding one type requires 9-11 edits; no user extensibility
  without forking
• TLS layer hacks - Tds.Tls GenServer has race conditions, unbounded buffers, and leaked processes
• Implicit state machine - Process.put for hidden state, unguarded transitions, silent transaction corruption
• No separation of concerns - packet framing, message construction, token parsing tangled together; binary
  macros in two modules with different endianness semantics
• Ad-hoc error handling - multiple SQL errors silently discarded, unknown tokens crash, no error classification

Phase 1: Constants & Binary Macros - Single Source of Truth

• Task 1.1 - Create Tds.Protocol.Constants (compile-time macros for all TDS protocol constants)
• Task 1.2 - Create Tds.Protocol.Binary (unified binary macros with explicit endianness)
• Task 1.3 - Migrate all modules to use Constants and Binary; fix prelogin :instopt decode bug
• Task 1.4 - Remove old BinaryUtils and Grammar modules

Phase 2: Packet Framing Layer

• Task 2.1 - Create Tds.Protocol.Packet (encode/decode TDS packet frames with bounded reassembly)
• Task 2.2 - Migrate Messages and Protocol to use Packet

Phase 3: Extensible Type System

• Task 3.1 - Define Tds.Type behaviour (type_codes, decode_metadata, decode, encode,
  param_descriptor, infer)
• Task 3.2 - Create Tds.Type.Registry (TDS code <-> handler module mapping, user types checked first)
• Task 3.3 - Implement built-in type handlers (integer, float, decimal, string, binary, boolean, datetime,
  uuid, money, xml)
• Task 3.4 - Wire Type.Registry into Protocol, replacing the monolithic types.ex

Phase 4: Transport Abstraction

• Task 4.1 - Create Tds.Transport behaviour and Transport.Tcp implementation
• Task 4.2 - Implement Transport.Tls + TlsHandshake cb_info module (eliminates GenServer, fixes race
  conditions)
• Task 4.3 - Migrate Protocol to use Transport

Phase 5: State Machine Cleanup

• Task 5.1 - Create Protocol.State with explicit phase transitions (eliminates Process.put, guards
  invalid transitions)
• Task 5.2 - Extract Protocol.Connection and Protocol.Execution modules

Phase 6: Error Handling

• Task 6.1 - Create error hierarchy (Tds.Error with :reason, :errors, :context, :original)
• Task 6.2 - Aggregate multiple SQL errors, add operation context, classify errors

Phase 7: Login/Prelogin & Connection Init Cleanup

• Task 7.1 - Fix Prelogin bugs (exhaustive encryption state match, replace inspect(self()) thread ID)
• Task 7.2 - Fix Login7 (input validation, automatic offset calculation, FeatureExt groundwork)
• Task 7.3 - Add session option validation with SQL injection prevention

Bonus

• Task B.1 - Create mix tds.export_errors task for refreshing SQL Server error codes CSV

Quality Gates

• All 269 existing tests pass after every commit
• Code coverage analysis to identify blind spots (currently 68% overall)
• Each new module has dedicated unit tests
• Compile clean: mix compile --warnings-as-errors
• Format clean: mix format --check-formatted
• No references to old modules: grep -r "BinaryUtils\|Protocol.Grammar" lib/ returns empty
• Public API unchanged: lib/tds.ex function signatures match v2.x
• Full integration tests against SQL Server via Docker