Skip to content

fix: generic type resolution for class methods and iterators#3392

Open
RomanSpector wants to merge 8 commits intoLuaLS:masterfrom
RomanSpector:fix/generic-class-resolution
Open

fix: generic type resolution for class methods and iterators#3392
RomanSpector wants to merge 8 commits intoLuaLS:masterfrom
RomanSpector:fix/generic-class-resolution

Conversation

@RomanSpector
Copy link
Copy Markdown

@RomanSpector RomanSpector commented Mar 28, 2026

Summary

Comprehensive fixes for generic type resolution in class methods, improving type inference for generic class patterns.

Problems Fixed

1. Generic class method return types not resolved (compiler.lua, sign.lua)

When calling a method on a generic class instance, return types using class generic parameters were not resolved:

---@class list<T>: {[integer]:T}

---@generic T
---@param self list<T>
---@param i intger
---@return T
function list:get(i) ... end

---@type list<string>
local strings
local v = strings:get(1) -- was: unknown, now: string

2. Nested generic resolution in doc.type.sign (generic.lua)

Generic parameters inside nested sign types (e.g. table<uint32, list<T>>) were not being cloned/resolved correctly, causing display issues like list<<T>> and incorrect type resolution.

3. ipairs(self) in generic class methods (sign.lua, generic.lua, compiler.lua)

Loop variables from ipairs(self) inside generic methods were not resolved:

---@class list<T>: {[integer]:T}

---@generic T
---@param self list<T>
function list:each()
    for i, v in ipairs(self) do
        -- v was: unknown, now: T
    end
end

The fix works through three layers:

  • sign.lua: Resolve V[] (from ipairs signature) against generic classes via class generic map, searching both @field annotations and extends tables ({[integer]:T})
  • generic.lua: Preserve generic-to-generic resolution wrappers (V->T) in mt:resolve when the resolved value is another generic parameter
  • compiler.lua: Allow resolved generic wrappers through the return type filter in bindReturnOfFunction

4. Double angle brackets in display (infer.lua)

Generic type parameters were displayed with double brackets: list<<T>> instead of list<T>. Fixed by stripping outer <> from generic name views when rendering doc.type.sign parameters, since the sign already wraps them.

5. Nil crash in invisible diagnostics (visible.lua)

getParentClass crashed with "attempt to index nil value" when a doc.field had no associated class. Added nil guard.

Test Plan

  • All existing tests pass (type_inference, hover, completion, diagnostics, full suite)
  • New tests for ipairs(self) with @field [integer] T pattern
  • New tests for ipairs(self) with @class list<T>: {[integer]:T} pattern
  • Updated display tests: table<K, V> instead of table<<K>, <V>>
  • Manual testing in VSCode with generic class code

RomanSpector and others added 7 commits March 27, 2026 22:57
- Fix type narrowing for `x == nil and "default" or x` pattern: the `and`
  handler in the tracer now propagates outNode when the right side is a
  truthy literal, so the `or` branch correctly infers the variable as non-nil
- Fix circular dependency in calcNode for assignments inside if/for blocks:
  a _compilingAssigns guard prevents incomplete types and stale marks from
  propagating when the if-handler's getNode(lastAssign) triggers recompilation
  of an already-compiling setlocal
- Filter no-unknown false positives when a node only contains a variable
  object whose base declaration has a known type

Closes LuaLS#2236, LuaLS#2374, LuaLS#2494
When iterating with ipairs(self) inside a generic method where self is
list<T>, the loop variable v was resolving to unknown instead of T.

The fix ensures the V->T resolution chain is preserved through three
layers: sign resolution finds the mapping via class generic map and
extends tables, generic.lua preserves the resolved wrapper when a
generic maps to another generic, and compiler.lua allows these
resolved-to-generic types through the return type filter.

Supports both @field [integer] T and @Class list<T>: {[integer]:T}
class definition patterns.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Strip outer <> from generic name views when rendering doc.type.sign
parameters, since the sign already wraps them in <>.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a doc.field has no associated class, getParentClass would pass
nil to getGlobalNode causing 'attempt to index a nil value' error
during invisible diagnostics.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements several fixes for type inference and generic resolution, specifically addressing type loss in control flow blocks due to circular dependencies and improving the inference of the x == nil and "default" or x idiom. It also enhances the handling of generic classes, arrays, and method return types, while preventing a nil crash in getParentClass and cleaning up generic type display formatting. Review feedback suggests documenting all major fixes in the changelog, extracting duplicated logic for identifying purely generic nodes into a shared utility, and refactoring deeply nested logic in script/vm/sign.lua for better maintainability.

Comment on lines +7 to +8
* `FIX` Fix type inference for `x == nil and "default" or x` idiom [#2236](https://github.com/LuaLS/lua-language-server/issues/2236)
* `FIX` Fix type loss for assignments inside `if`/`for` blocks due to circular dependency in tracer [#2374](https://github.com/LuaLS/lua-language-server/issues/2374) [#2494](https://github.com/LuaLS/lua-language-server/issues/2494)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The changelog entries are for secondary fixes in this PR. Please also add entries for the main fixes related to generic type resolution as described in the PR summary. For example:

  • FIX Resolve generic class method return types.
  • FIX Correctly resolve nested generics in doc.type.sign.
  • FIX Resolve loop variables for ipairs(self) in generic methods.
  • FIX Fix display of generic types with double angle brackets (e.g., list<<T>>).
  • FIX Prevent nil crash in getParentClass for doc.field without a class.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Added changelog entries for all main generic resolution fixes.

Comment on lines +1675 to +1681
local allGeneric = true
for rn in clonedObject._resolved:eachObject() do
if rn.type ~= 'doc.generic.name' then
allGeneric = false
break
end
end
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This logic to check if a resolved node contains only generic names is duplicated in this file (lines 1796-1801) and also in script/vm/generic.lua (lines 203-209). To improve maintainability and reduce duplication, consider extracting this into a helper function in a shared module, for example on the vm object.

For example, you could define a function in vm.vm:

function vm.isPurelyGenericNode(node)
    if not node then return false end
    for rn in node:eachObject() do
        if rn.type ~= 'doc.generic.name' then
            return false
        end
    end
    return true
end

Then you could simplify this block and the other occurrences by calling vm.isPurelyGenericNode(...).

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Extracted vm.isResolvedToGeneric() helper in generic.lua to replace all 3 duplicated checks.

Comment on lines +103 to +136
vm.getClassFields(uri, classGlobal, vm.declareGlobal('type', 'integer'), function (field)
if field.extends then
guide.eachSourceType(field.extends, 'doc.generic.name', function (src)
if genericMap[src[1]] then
resolved[vKey] = genericMap[src[1]]
handled = true
end
end)
end
end)
-- Also search extends tables (for @class list<T>: {[integer]:T})
if not handled then
for _, set in ipairs(classGlobal:getSets(uri)) do
if set.type == 'doc.class' and set.extends then
for _, ext in ipairs(set.extends) do
if ext.type == 'doc.type.table' and ext.fields then
for _, field in ipairs(ext.fields) do
if field.extends then
guide.eachSourceType(field.extends, 'doc.generic.name', function (src)
if genericMap[src[1]] then
resolved[vKey] = genericMap[src[1]]
handled = true
end
end)
end
if handled then break end
end
end
if handled then break end
end
end
if handled then break end
end
end
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This block of code has a couple of areas for improvement:

  1. Deep Nesting: The logic to search extends tables (lines 114-136) is nested up to 6 levels deep. This reduces readability and makes the code harder to maintain. Consider refactoring this into smaller helper functions to reduce the nesting depth.

  2. Inefficient Loops: The callbacks to guide.eachSourceType (lines 105-110 and 121-126) set handled = true, but the loops continue to iterate. If guide.eachSourceType can be made to break early (e.g., by returning a value from the callback), it would improve efficiency.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Extracted findGenericInTableFields() and simplified findGenericInExtendsTable() — max nesting reduced from 6 to 3 levels.

- Extract vm.isResolvedToGeneric() helper to replace duplicated
  allGeneric checks in generic.lua, compiler.lua (3 locations)
- Extract findGenericInTableFields() and simplify
  findGenericInExtendsTable() in sign.lua to reduce nesting
- Add changelog entries for generic resolution fixes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@RomanSpector RomanSpector force-pushed the fix/generic-class-resolution branch from f322cdc to 5c78df1 Compare March 28, 2026 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant