NA dimensions for complex and tuple types

[SteveBronder]

Describe the bug

When calling $draws() on a fit object whose model has complex or tuple types in it I receive warning messages about NA's being made for the variable sizes.

To Reproduce

I made the stan program below to test this

parameters {
  real real_p;
  vector[2] vector_p;
  matrix[2, 2] matrix_p;
  array[2] matrix[2, 2] array_matrix_p;
  corr_matrix[2] corr_p;
  complex complex_p;
  complex_matrix[2, 2] complex_matrix_p;
  complex_vector[4] complex_vector_p;
  tuple(real, vector[3], array[2] matrix[2, 2], complex) tuple_int_vector_arraymatrix_complex_p;
  array[2] tuple(real, tuple(vector[2], array[2] tuple(real, complex, matrix[2, 2]))) arraytuple_big_p;
}

You can get the error with the following

# I put this in the tests/testthat/resources/stan folder
mod_params <- testing_model("parameter_types")
fit <- mod_params$pathfinder(data=data_list, seed=1234, refresh = 0)
foo = fit$draws(format = "df")
# Warning messages:
# 1: In variable_dims(metadata$variables) : NAs introduced by coercion
# 2: In variable_dims(metadata$variables) : NAs introduced by coercion
# 3: In variable_dims(metadata$variables) : NAs introduced by coercion
# 4: In variable_dims(metadata$variables) : NAs introduced by coercion
# 5: In variable_dims(metadata$variables) : NAs introduced by coercion
meta = fit$metadata()
meta$stan_variable_sizes
## Output
$lp_approx__
[1] 1

$lp__
[1] 1

$real_p
[1] 1

$vector_p
[1] 2

$matrix_p
[1] 2 2

$array_matrix_p
[1] 2 2 2

$corr_p
[1] 2 2

$complex_p
[1] NA

$complex_matrix_p
[1]  2  2 NA

$complex_vector_p
[1]  4 NA

$`tuple_int_vector_arraymatrix_complex_p:1`
[1] 1

$`tuple_int_vector_arraymatrix_complex_p:2`
[1] 3

$`tuple_int_vector_arraymatrix_complex_p:3`
[1] 2 2 2

$`tuple_int_vector_arraymatrix_complex_p:4`
[1] NA

$arraytuple_big_p
[1] NA NA  2  2

Expected behavior

Tuples and complex types would have the correct size

Operating system
Linux redhat

CmdStanR version number
master 28329d7

Additional context
I found this while working on the functions for adding pathfinder types

[avehtari]

Root Cause Analysis

The warning NAs introduced by coercion comes from the variable_dims function in csv.R (line ~1003: dims[[var]] <- as.numeric(var_indices)).

Issue 1: Complex numbers (z.real, z.imag)

CmdStan outputs complex number columns as z.real and z.imag. The repair_variable_names function converts . to [ and ], turning these into z[real] and z[imag]. Then variable_dims strips the ], gets "real" and "imag" as index strings, tries to parse the last one ("imag") with as.numeric(), and gets NA.

Issue 2: Tuples (d_tuple:1, d_tuple:2.1, etc.)

CmdStan uses : as a separator for tuple element indexing (e.g., d_tuple:1, d_tuple:2.1, d_tuple:3.1.1). The repair_variable_names function doesn't understand the : separator — it only handles . → [,] conversion. So d_tuple:1 stays as d_tuple:1 (no dots, no brackets). Then variable_dims computes:

• Unique base names: d_tuple:1, d_tuple:2, d_tuple:3, z
• For d_tuple:1: the pattern ^d_tuple:1\[ matches nothing, so it becomes a scalar with dim 1 — this is wrong, it loses the tuple structure, but doesn't cause the NA warning itself.

The NA warning is specifically caused by the complex number variable z, where "imag" cannot be coerced to numeric. The warning appearing twice corresponds to the two calls to variable_dims(metadata$variables) — once during initial CSV reading (line 328), and once lazily when print(fit) is first called (which triggers metadata reading for a different purpose).

Summary

Both repair_variable_names and variable_dims in csv.R don't handle two new CmdStan output formats:

1. Complex numbers: columns named <var>.real / <var>.imag — the non-numeric suffixes real/imag cause as.numeric() to produce NA in variable_dims.
2. Tuples: columns using : as a separator (e.g., d_tuple:1, d_tuple:2.1) — repair_variable_names doesn't recognize : and the tuple structure is silently misrepresented (each tuple element becomes a separate "variable" instead of being grouped).

[avehtari]

The posterior package correctly parses both the : tuple separator and the complex number .real/.imag suffixes, so the problem is only CmdStanR showing warnings and NAs in CmdStanR metadata (metadata$stan_variable_sizes, metadata$stan_variables)

[avehtari]

Plan which shows that this bug has other effects than just warnings

Task 1: Fix repair_variable_names / unrepair_variable_names

The entry point. CmdStan now uses three separators in CSV headers:

Separator	Meaning	Example raw	Correct repaired
. between digits	array index	beta.1.2	beta[1,2]
:	tuple element	b_tuple:1:1.2	b_tuple:1:1[2]
.real / .imag	complex part	z.real	z[real]

Currently the function blindly converts every . → [,], which happens to work for tuples (the : part has no dots) but breaks for complex (.real becomes [real] which is fine for posterior but causes variable_dims to choke on non-numeric index).

Task 2: Fix variable_dims for complex numbers

The z variable gets NA dimension because as.numeric("imag") fails. Should produce $z = 2. Tuples are actually fine here — each leaf like b_tuple:1:1 is correctly treated as its own variable.

Task 3: Fix create_skeleton for tuples

create_skeleton() looks up param_metadata_[target_params] where:

• target_params = Stan-level names from model_variables: b_tuple, z, etc.
• param_metadata_ = C++ flattened leaf names: b_tuple.1.1, b_tuple.1.2, b_tuple.2, z

The b_tuple lookup fails → NA name in skeleton. The fix must expand tuple names from model_variables into their leaf components from param_metadata_.

Task 4: Fix unconstrain_draws / unconstrain_variables for tuples

These use stan_variables (leaf names with : like b_tuple:1:1) to filter against model_variables$parameters (Stan-level names like b_tuple). The %in% check fails for tuples → tuple parameters silently dropped → CmdStan gets wrong number of scalars → crash.

Task 5: Tests

[jgabry]

Thanks Aki. This diagnosis of the problem makes sense (thank you Claude, I assume). Can Claude also implement this plan?

[avehtari]

The diagnosis is Claude assisted, but as usual it required clear guidance and good examples. Claude probably can assist in implementation, but I'd expect it require also substantial human input and time. I need to work on other things, but maybe someday I have time for this.

[jgabry]

I need to work on other things, but maybe someday I have time for this.

I see #1174. Seems like you had time ;) Thank you!