Collate-free import sorting

[jakebailey]

We currently import x/text/collate to implement import sorting. But, this has cost:

• The tables are huge. They add 1.3MB to our binary.
• The tables are very out of date. They're on CLDR 23, which was for Unicode 6.3!

While I think I can fix the latter with a CL stack I've been trying, the former is just unfixable.

I do not think anyone out there is actually writing non-ASCII imports, but the case sensitivity and "natural" sorting is useful. It turns out you can get really close to the old behavior with just the stdlib and x/text/norm for accent normalization.

My proposal is this:

• Create a new import sorting scheme that does not require collation.
• Transparently use the new import sorting scheme that matches best for the old options.
• Deprecate the old options.

As mentioned before, this drops 1.3MB out of our binary.

The new options are:

type OrganizeImportsSort int

const (
	OrganizeImportsSortAuto OrganizeImportsSort = iota
	OrganizeImportsSortOrdinal
	OrganizeImportsSortOrdinalIgnoreCase
	OrganizeImportsSortNatural
	OrganizeImportsSortNaturalIgnoreCase
)

Which do what they say on the tin.

In terms of which tools are closest to which option:

• eslint-plugin-simple-import-sort: naturalIgnoreCase.
• dprint: ordinal or ordinalIgnoreCase.
• ESLint's built-in sort-imports: ordinal or ordinalIgnoreCase.
• eslint-plugin-import/order: ordinal or ordinalIgnoreCase.
• @trivago/prettier-plugin-sort-imports / @ianvs/prettier-plugin-sort-imports: natural or naturalIgnoreCase
• biome: natural or ordinal
• oxfmt: natural or naturalIgnoreCase
• eslint-plugin-perfectionist/sort-imports: in natural mode, natural or naturalIgnoreCase (plus a mode that is more like ordinalIgnoreCase but it's actually system locale dependent, ouch)

[jakebailey]

You'll note that no tests change behavior with this mapping; no manual test changes etc etc. This gives me hope!

[jakebailey]

Okay, almost, see #4349 (comment)

[jakebailey]

This is here solely to split letters from their accents for case insensitive sorting. We do test this.

Unfortunately, this is more tables internally, also out of date.

[Copilot]

Pull request overview

This PR replaces the existing import-sorting implementation that depended on x/text/collate with a collate-free approach, introducing a new OrganizeImportsSort preset enum and wiring it through LSP preferences, tests, and fourslash fixtures/baselines.

Changes:

• Added OrganizeImportsSort preference (including parsing/serialization) and updated config/baselines to emit "sort": "auto".
• Reimplemented organize-import string comparison without collate, including natural/numeric/diacritic-aware comparisons and updated sorting/detection logic.
• Updated LSP/session tests and fourslash tests (and conversion script) to use the new organizeImportsSort preference.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
testdata/baselines/reference/fourslash/state/codeLensAcrossProjects.baseline	Baseline update to include new organizeImports.sort field.
internal/project/session_test.go	Updates session preference parsing expectations to use organizeImportsSort.
internal/ls/organizeimports.go	Integrates resolved sort mode into organize-imports comparer selection/detection.
internal/ls/lsutil/utilities_test.go	Adds tests for sort resolution and natural string comparison behavior.
internal/ls/lsutil/userpreferences.go	Adds OrganizeImportsSort enum + config plumbing; refactors raw-field config handling.
internal/ls/lsutil/userpreferences_test.go	Extends preference parsing tests for old/new organize-import settings.
internal/ls/lsutil/organizeimports.go	Removes collate-based comparer; adds collate-free comparison and detection updates.
internal/fourslash/tests/organizeImports_sortModuleSpecifiers_test.go	Switches fourslash preferences to OrganizeImportsSortOrdinalIgnoreCase.
internal/fourslash/tests/organizeImports_coalesceImports_test.go	Switches fourslash preferences to OrganizeImportsSortOrdinalIgnoreCase.
internal/fourslash/tests/organizeImports_coalesceExports_test.go	Switches fourslash preferences to OrganizeImportsSortOrdinalIgnoreCase.
internal/fourslash/_scripts/convertFourslash.mts	Teaches converter to map organizeImportsSort into Go preferences.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

[jakebailey]

Strada actually did this wrong, which is terrible