Remove Generic-based Flat derivation machinery#7662
Conversation
6825737 to
4f79329
Compare
409fb17 to
5b0a29e
Compare
5b0a29e to
a845510
Compare
The internal flat library contained Generic-based Flat class derivation via GHC.Generics that had a bug: deserializing large enum types (512+ constructors) consumed infinite memory. While not a security risk (UPLC on-chain code uses only manual Flat instances), the buggy dead code was distracting. This commit removes the entire Generic derivation machinery. Changes: 1. Add manual Flat instances to all types that previously relied on Generic defaults: Maybe, Either, Complex, [], NonEmpty, tuples (2-7), Tree, Filler, PostAligned, PreAligned, SrcSpan, SrcSpans, and all PIR types (Recursivity, Strictness, Datatype, Binding, Program, Term). Manual encoding is bit-identical to the Generic encoding. 2. Add missing `size` methods to partial Flat instances in FlatInstances.hs (Name, Version, VarDecl, TyVarDecl, Program, NamedDeBruijn) and Value.hs (K, Quantity, Value) that previously relied on the Generic default for `size`. 3. Gut Class.hs: remove ~440 lines of GFlat*/GEncode*/GSize* classes, NumConstructors type family, all Generic default method implementations, and associated imports/pragmas/re-exports. 4. Remove ENUM_LARGE test infrastructure (E256/E258 types, Flat/Arbitrary instances, ~530 lines) and delete Core.hs inspection tests. 5. Fix transitive import breakage: files that got Generic from the PlutusCore.Flat re-export now import GHC.Generics directly. Remove `hiding (to)` from imports that no longer re-export Generics. 6. Strip Generic-specific doctests from Tutorial.hs. All existing tests pass (5281 across 4 suites). Manual instances produce bit-identical encoding verified by round-trip tests and golden files.
Mutation testing on the hand-written Flat instances from #7542 found 9 coverage gaps (none on the on-chain path). Add roundtrip tests for Tree, Set, PreAligned, PIR Program, SrcSpan, SrcSpans, and NamedDeBruijn. Remove the dead OVERLAPPING Flat [Char] instance from Extra.hs — the module was never imported by the umbrella module, so GHC always resolved Flat String to the generic [a] instance.
a845510 to
6c8826f
Compare
Mutation testing on Flat instancesRan mutation testing on all 32 hand-written Flat instances from this PR to check test coverage. For each instance, I replaced Initial run: 22/32 killed, 10 survivedThe first pass found 10 uncovered instances. None of them were on the on-chain serialization path, but some were worth covering anyway:
Added tests, re-ran: 31/32 killedI added roundtrip tests for every survivor and re-ran. All 9 are now killed. The sole remaining survivor is Tests added in this follow-up:
Full results table (click to expand)
Dead code removed:
|
Summary
Closes #7542.
The internal
flatlibrary contained ~440 lines of Generic-basedFlatclass derivation machinery (GFlat*,GEncode*,GSize*classes,NumConstructorstype family) that had a known bug: deserializing enum types with 512+ constructors consumed infinite memory. While not a security risk (all on-chain UPLC code uses only manualFlatinstances), the buggy dead code was distracting. This PR removes the entire Generic derivation machinery.Changes
Add manual
Flatinstances to all types that previously relied on Generic defaults:Maybe,Either,Complex,[],[Char],NonEmpty, tuples (2-7),Tree,Filler,PostAligned,PreAligned,SrcSpan,SrcSpans, and all PIR types (Recursivity,Strictness,Datatype,Binding,Program,Term). Manual encoding is bit-identical to the Generic encoding.Add missing
sizemethods to 9 partialFlatinstances inFlatInstances.hs(Name,Version,VarDecl,TyVarDecl,Program,NamedDeBruijn) andValue.hs(K,Quantity,Value) that previously relied on the Generic default forsize.Gut
Class.hs: remove ~440 lines of Generic machinery, leaving only the abstractFlatclass withencode,decode,sizemethods andgetSize.Remove
ENUM_LARGEtest infrastructure:E256/E258types, theirFlat/Arbitraryinstances (~530 lines), and the-DENUM_LARGEcpp-option.Delete
Core.hsandTutorial.hs:Core.hscontained GHC inspection tests verifying that Generic-derived code optimized well, andTutorial.hstaught Flat usage viaderiving (Generic, Flat)examples — both are meaningless without Generic derivation, and the original tutorial remains available in the upstreamflatpackage on Hackage.Fix transitive import breakage: files that got
Genericfrom thePlutusCore.Flatre-export now importGHC.Genericsdirectly. Remove stalehiding (to, from)from imports that no longer re-export Generics.Add QuickCheck round-trip property test for PIR Flat instances: generates 200 random well-typed PIR terms via
genTypeAndTerm_, encodes withflat, decodes withunflat, and verifies byte-level identity of re-encoded output.