rustc hangs when compiling dyn-encoded polymorphic recursion

[naganohara-yoshino]

Summary

rustc appears to hang indefinitely when compiling a generic function that performs polymorphic recursion over a non-regular recursive type pattern.

The core issue is that the recursive call changes the type parameter from A to (A, A). Conceptually, this leads to an infinite sequence of distinct instantiations:

myprint::<A>
myprint::<(A, A)>
myprint::<((A, A), (A, A))>
myprint::<(((A, A), (A, A)), ((A, A), (A, A)))>
...

I believe this should terminate with a diagnostic, such as a recursion-limit, overflow, or infinite monomorphization error, rather than hanging during compilation.

Code

I tried this code:

#![allow(dead_code)]

use std::fmt::Debug;

enum PerfectPrime<A> {
    Fst(A),
    Snd(Box<dyn Fn() -> PerfectPrime<(A, A)>>),
}

impl<A> PerfectPrime<A> {
    fn fst(a: A) -> Self {
        PerfectPrime::Fst(a)
    }
}

fn myprint<A: Debug>(p: PerfectPrime<A>) {
    match p {
        PerfectPrime::Fst(a) => println!("{:?}", a),
        PerfectPrime::Snd(f) => myprint(f()),
    }
}

fn main() {
    let p0: PerfectPrime<i32> = PerfectPrime::fst(42);

    myprint(p0);
}

Expected behavior

rustc should terminate with a diagnostic.

The program uses a non-regular recursive type pattern:

enum PerfectPrime<A> {
    Fst(A),
    Snd(Box<dyn Fn() -> PerfectPrime<(A, A)>>),
}

The recursive occurrence is not at the same type parameter A, but at (A, A). This is analogous to a non-regular recursive type in OCaml, such as:

type 'a perfect =
  | Fst of 'a
  | Snd of ('a * 'a) perfect

or, more precisely for this Rust example, a lazy/thunked version:

type 'a perfect_prime =
  | Fst of 'a
  | Snd of (unit -> ('a * 'a) perfect_prime)

The dyn Fn indirection allows the type definition itself to avoid the immediate drop-check / well-formedness overflow that would happen with a direct recursive field. However, the function myprint then performs polymorphic recursion:

fn myprint<A: Debug>(p: PerfectPrime<A>) {
    match p {
        PerfectPrime::Fst(a) => println!("{:?}", a),
        PerfectPrime::Snd(f) => myprint(f()),
    }
}

In the Snd branch, f() has type:

PerfectPrime<(A, A)>

Therefore, the recursive call is not:

myprint::<A>(...)

but instead:

myprint::<(A, A)>(...)

This appears to force the compiler to consider an infinite sequence of distinct instantiations:

myprint::<A>
myprint::<(A, A)>
myprint::<((A, A), (A, A))>
myprint::<(((A, A), (A, A)), ((A, A), (A, A)))>
...

Since Rust monomorphizes generic functions, this recursion does not correspond to a single compiled instance. It expands into infinitely many different type instances.

I expected rustc to detect this and report an error, for example a recursion-limit, overflow, or infinite monomorphization diagnostic.

Actual behavior

rustc appears to hang during compilation.

In my local test, the following command did not finish after 1 hour 1 minute 15 seconds:

cargo run --package app --bin a

The output stayed at:

Compiling app v0.1.0 (F:\Coding\rs-projects\lab\app)
Building [                             ] 0/1: a(bin)

No diagnostic was emitted.

Why this seems notable

A directly recursive non-regular version such as:

enum Perfect<A> {
    Fst(A),
    Snd(Box<Perfect<(A, A)>>),
}

does fail with an overflow diagnostic, for example during drop-check rule generation.

However, the thunked version:

enum PerfectPrime<A> {
    Fst(A),
    Snd(Box<dyn Fn() -> PerfectPrime<(A, A)>>),
}

avoids that earlier structural recursion check because the recursive value appears behind a trait object return type. The type definition itself is accepted.

The non-regular recursion then reappears at the function level when myprint recursively calls itself at a different type parameter. This seems to expose a conflict between:

1. non-regular recursive type patterns encoded through a dynamic thunk, and
2. Rust's monomorphization strategy for generic functions.

The compiler appears to keep generating or reasoning about larger and larger instantiations instead of producing a bounded diagnostic.

This is interesting because the analogous program in OCaml or Haskell would be accepted as ordinary polymorphic recursion over a non-regular recursive type. The recursive call is type-correct: it calls the same polymorphic function at (A, A) instead of A.

In Rust, however, the same pattern interacts poorly with monomorphization. Even though the concrete value passed in main is Fst(42) and the Snd branch is never reached at runtime, rustc still has to compile the generic function body. That body recursively refers to myprint::<(A, A)>, then myprint::<((A, A), (A, A))>, and so on.

Meta

rustc --version --verbose:

rustc 1.97.0-nightly (e95e73209 2026-05-05)
binary: rustc
commit-hash: e95e73209faf6ead2bc5c7636e45e589a751b79b
commit-date: 2026-05-05
host: x86_64-pc-windows-msvc
release: 1.97.0-nightly
LLVM version: 22.1.4

No backtrace is available because compilation does not complete and no panic is emitted.