Have framework for proxying serial connection capabilities established

[mkerstner]

Problem statement

Serial proxying should become a first class citizen and should feel approachable by integrators to use and build upon in a consistent manner.

• This bet tackles the first steps of our roadmap opportunity of Improve proxying serial connection capabilities #23

Scope & Boundaries

In scope

• Essentially Epics 1-3 of the roadmap opportunity Improve proxying serial connection capabilities #23
• Defining a serial (proxy) protocol
• Providing framework for building proxying integrations to build stable serial communication around it

Not in scope

• Individual Home Assistant integrations
• Instead focus on providing a serial proxying framework

Foreseen solution

Serial proxy protocol implementation as part of ESPHome and Home Assistant integration framework available.

Community signals

This opportunity addresses long-standing community interest in improving connectivity through proxying serial communication.

Risks & open questions

No response

Appetite

Medium - 4-6 weeks

Execution issues

No response

Decision log

Date	Decision	Outcome

[puddly]

The low-level side is done:

• The ESPHome side: [serial_proxy] New component esphome/esphome#13944
• The ESPHome client library side: [serial_proxy] Add awaitable serial proxy subscribe/unsubscribe responses esphome/aioesphomeapi#1537
• The library to communicate with ESPHome as a serial port: https://github.com/puddly/serialx/releases/tag/v1.1.1

The Home Assistant backend side:

• Replace PySerial with serialx internally within Home Assistant Core: Switch USB integration to list serial ports with serialx home-assistant/core#167615
• Fix all integrations that erroneously use PySerial to list serial ports to instead use our USB integration's helpers instead.
• Introduce package --excludes to prevent integrations and custom components from using PySerial.
• Provide an API-compatible replacement package for pyserial, pyserial-asyncio, and pyserial-asyncio-fast: https://pypi.org/project/serialx-compat/
  • This allows existing integrations communicating over serial to transparently use esphome://... URIs.

The more user-friendly parts in Home Assistant will require a bit more alignment (e.g. modbus) but the above three tasks should wrap up the backend side of things.