-
Notifications
You must be signed in to change notification settings - Fork 317
Description
Problem
We pin protobuf to ~> 0.13.0 in packages/sync-service/mix.exs to avoid a duplicate module conflict at mix release time. The conflict is between Google.Protobuf.* well-known type modules shipped by two different protobuf libraries in our dependency tree:
protobuf(>= 0.14.0) — used byotel_metric_exporterfor OTLP metric/log encoding. Starting with v0.14.0, it bundlesGoogle.Protobuf.*modules (previously in the separategoogle_protospackage).protox— used bypg_query_exfor Postgres parse tree (de)serialization. Its well-known type source files (lib/google/protobuf/*.ex) defineProtox.Google.Protobuf.*container modules, but theuse Protox.Definemacro inside them generates bareGoogle.Protobuf.*modules at compile time (viaDefineMessage.define/2which doesdefmodule unquote(msg_schema.name)wherename:isGoogle.Protobuf.Any, notProtox.Google.Protobuf.Any).
The conflict only manifests during mix release (not mix compile), because that's when OTP checks for duplicate modules across applications.
Neither side's Google.Protobuf.* modules are actually used by our code — otel_metric_exporter's generated .pb.ex files are namespaced under OtelMetricExporter.*, and pg_query_ex's proto schema has no well-known type imports.
Approaches explored
1. Remove conflicting source files in Dockerfile (quick fix)
Add RUN rm -rf deps/protox/lib/google between mix deps.get and mix deps.compile in the Dockerfile. This prevents protox from compiling the well-known type modules, and Mix auto-generates a clean .app file without them.
Verified locally: protox compiles fine (32 files instead of 50) and mix release passes.
Pros: minimal, no upstream changes needed.
Cons: only applies to Docker builds; local mix release would still fail without a similar step.
2. Fork protox and delete lib/google/
Fork ahamez/protox, remove the lib/google/ directory, point pg_query_ex at the fork.
Pros: clean, works everywhere (not just Docker).
Cons: fork maintenance burden.
3. Post-compile cleanup in mix.exs
Add a Mix alias that removes Google.Protobuf.*.beam files from protox's ebin/ and patches its .app file after compilation.
Pros: works for all build contexts.
Cons: fragile, modifies compiled artifacts.
4. Fix protox upstream
Change protox's well-known type schemas to use Protox.Google.Protobuf.* as the module name: so DefineMessage generates prefixed modules. Requires updating parse.ex internals and the code generator's handling of well-known type references. This is a breaking change for protox.
Pros: proper fix at the source.
Cons: breaking change, significant scope, depends on upstream maintainer.
5. Unify on a single protobuf library (preferred direction)
Use either protobuf or protox in both otel_metric_exporter and pg_query_ex, eliminating the conflict entirely.
- Option A: Migrate
pg_query_exto useprotobuf— regenerate the proto files withprotoc-gen-elixir(from theprotobufpackage) instead ofprotox.pg_query_exis our own package so we control this. - Option B: Migrate
otel_metric_exporterto useprotox— regenerate the OTLP proto files withprotox. Also our own package.
Pros: eliminates the root cause, no workarounds needed, one fewer dependency in the tree.
Cons: requires regenerating proto code and updating encode/decode call sites.