Skip to content

Support MATLAB-style leading-dot float literals#216

Closed
nallana wants to merge 1 commit intomainfrom
codex/linear-mention-rm-34-support-matlab-compatible-float-liter
Closed

Support MATLAB-style leading-dot float literals#216
nallana wants to merge 1 commit intomainfrom
codex/linear-mention-rm-34-support-matlab-compatible-float-liter

Conversation

@nallana
Copy link
Copy Markdown
Contributor

@nallana nallana commented Mar 23, 2026

Motivation

  • The lexer currently rejects MATLAB-style numeric literals that omit the leading zero (e.g., .1), causing parsing failures in normal scripts and range expressions like 0:.1:10.
  • The fix must be done in the tokenization layer to be robust and avoid brittle downstream hacks while preserving existing dot-based syntax (obj.field, 1..3, .*, etc.).

Description

  • Teach tokenize_detailed() in crates/runmat-lexer/src/lib.rs to contextually merge a standalone . into a Token::Float when it is followed by a valid fractional/exponent suffix, by detecting Token::Dot and scanning the remainder for a fractional/exponent sequence and underscores as digit separators.
  • Add helpers can_start_leading_dot_float, leading_dot_float_suffix_len, scan_decimal_digits, and scan_float_exponent to safely identify valid leading-dot float suffixes and ensure the dot is not being used for adjacent member/transpose-style syntax.
  • Preserve existing behavior for dot-based operators and property access (e.g., obj.field, .*, 1..3) and normalize underscore-separated digits in the emitted float lexeme consistently with other numeric tokens.
  • Add regression tests: lexer tests in crates/runmat-lexer/tests/lexer.rs, parser tests in crates/runmat-parser/tests/range_parsing.rs, and runtime/ignition test in crates/runmat-ignition/tests/ranges.rs to cover standalone leading-dot literals, range-step forms like 0:.1:1, and ensure property-access dots are unchanged.

Testing

  • Ran cargo fmt to ensure formatting completed successfully.
  • Ran cargo test -p runmat-lexer --test lexer and all lexer tests (including new leading-dot tests) passed.
  • Ran cargo test -p runmat-parser --test range_parsing and parser range tests (including leading-dot forms) passed.
  • Ran cargo test -p runmat-ignition --test ranges and runtime/ignition range tests (including a leading-dot step execution test) passed.

Codex Task


Note

Medium Risk
Touches core lexing behavior by rewriting . into Float in specific contexts, which could subtly affect dot-based syntax (property access/operators) if the heuristics are wrong; changes are guarded and covered by new tests.

Overview
Adds MATLAB-style leading-dot float support (e.g., .1, .25e-2, .3_5) by teaching tokenize_detailed() to contextually merge a standalone Token::Dot plus a following digit/exponent sequence into a single Token::Float (with underscore normalization), while avoiding rewrites when the dot is adjacent to a prior value/transpose.

Extends test coverage across lexer, parser, and runtime execution to ensure leading-dot floats work in assignments and range steps (e.g., 0:.1:1) and that obj.field-style property access remains tokenized as Ident Dot Ident.

Written by Cursor Bugbot for commit ee12f68. This will update automatically on new commits. Configure here.

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 23, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
runmat Ready Ready Preview, Comment, Open in v0 Mar 23, 2026 10:33pm

Request Review

Copy link
Copy Markdown
Contributor

@gneeri gneeri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to fix the clippy things that are causing CI to fail, but otherwise, looks good to me.

@nallana nallana closed this Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support MATLAB-compatible float literals without leading zero

2 participants