Skip to content

Syllable spacing in Lua#1720

Open
davidweichiang wants to merge 28 commits into
gregorio-project:developfrom
davidweichiang:lua-syllable-spacing
Open

Syllable spacing in Lua#1720
davidweichiang wants to merge 28 commits into
gregorio-project:developfrom
davidweichiang:lua-syllable-spacing

Conversation

@davidweichiang
Copy link
Copy Markdown
Contributor

@davidweichiang davidweichiang commented Mar 1, 2026

This is aiming towards #1673, but pursuing a different strategy than what was proposed there.

The first commit (02ec952) moves syllable rewriting into Lua, which has the following benefits:

  • The old algorithm worked by moving the last part of a syllable to the first part of the next syllable. It could therefore break ligatures between the middle and last part of a syllable (Syllable rewriting bugs #1719). The new algorithm works by joining the text boxes together; it does not break ligatures.
  • If maximumspacewithoutdash > 0, the old algorithm could make the last part of a syllable "jump" across the space between syllables (Syllable rewriting bugs #1719 again); the new one only rewrites syllables if they are actually touching.

ETA: These two cases are demonstrated in gregorio-project/gregorio-test#420.

And there are still the following problems:

  • The old algorithm computed syllable spacing after rewriting; the new algorithm computes them in the opposite order. This means that some syllable spacing will be slightly too big or small.
  • When the width of a syllable changes due to rewriting, the new syllable is left-aligned with the old syllable, but it might make more sense to center it with the old syllable.

The first problem can be seen in the test
tex-output/syllable-rewriting/syllable-rewriting.tex

ETA: The existing code was already making an approximation, in that it was aligning the first note and vowel without taking ligaturing and kerning into account. The new code is just extending this approximation to possibly a whole word. In my opinion, the approximation is visible only in the test syllable-rewriting.tex (because of the unusual font) and causes only a miniscule difference elsewhere.

Further tests that have miniscule differences:
gabc-output/glyphs/torculus_resupinus_quilisma.gabc
gabc-output/glyphs/torculus_resupinus.gabc
gabc-output/slurs.gabc
gabc-output/FactusEst.gabc
tex-output/dominican-flats/dominican-flats.tex
tex-output/SalveReginaOP/SalveReginaOP.tex
tex-output/bugs/fix-595/fix-595.tex
tex-output/bugs/fix-1110/fix-1110.tex
tex-output/lyric-centering/lyric-centering.tex

Tests that have big changes:

Closes #1719.

…erently from the old one, in the following ways:

+ The old algorithm worked by moving the last part of a syllable to the first part of the next syllable. It could therefore break ligatures between the middle and last part of a syllable. The new algorithm works by joining the text boxes together; it does not break ligatures.
+ If maximumspacewithoutdash > 0, the old algorithm could make the last part of a syllable "jump" across the space between syllables; the new one only rewrites syllables if they are actually touching.
- The old algorithm computed syllable spacing after rewriting; the new algorithm computes them in the opposite order. This means that some syllable spacing will be slightly too big or small. This will be fixable when syllable spacing is also moved to Lua.
…calculated in TeX for now, because it's needed in a couple of places, but \GreSyllable only generates a zero-width \hskip, and the Lua code recalculates it and resizes the \hskip accordingly.

The Lua syllablefinalskip calculation uses the actual begindifference of the next syllable, not the predicted one (nextbegindifference). In the old code, sometimes this prediction was wrong (gregorio-project#1723). Therefore this commit breaks several tests, but I believe they are all actually bug fixes.
@davidweichiang
Copy link
Copy Markdown
Contributor Author

davidweichiang commented Mar 6, 2026

Commit 4fa508d starts to move the calculation of syllablefinalskip into Lua. The TeX code computes the minimum distance between text and notes (between \GreSyllables only, not \GreBarSyllables), and emits a zero-width skip instead of syllablefinalskip. The Lua code computes the syllablefinalskip and edits the skip to have the right width.

Because it uses the actual begindifference of the next syllable, not nextbegindifference, and there are some existing bugs in how nextbegindifference is calculated, there are quite a few new broken tests.

Because of #1723:
gabc-output/glyphs/accidentals.gabc (case 1)
gabc-output/glyphs/brackets.gabc (case 4)
gabc-output/bugs/fix-1700.gabc (case 2)
gabc-output/soft-alterations.gabc (case 3)
tex-output/nabc-alignment/nabc-alignment-two-voices.tex (case 5)
tex-output/nabc-alignment/nabc-alignment.tex (case 5)
gabc-output/lines/2-lines.gabc (case 6)
gabc-output/lines/3-lines.gabc (case 6)
gabc-output/lines/5-lines.gabc (case 6)

Because of #1724:
gabc-output/glyphs/clef_change.gabc
gabc-output/glyphs/only-special.gabc
gabc-output/bugs/fix-1285.gabc
gabc-output/edge_cases.gabc

Only miniscule changes:
gabc-output/glyphs/punctum-inclinatum-2.gabc

- Move syllable clearing for \GreSyllables into Lua, which works by directly measuring the distances between syllables. This fixes issue gregorio-project#1725 but also breaks tests/gabc-output/glyphs/clear.gabc because it now inserts no extra space (and never removes space).
- Simplify syllable spacing by directly measuring distances between syllables, similarly to above.
@davidweichiang
Copy link
Copy Markdown
Contributor Author

Commit ec4e22c moves syllable clearing into Lua.

The amount to shift a syllable in order to clear it is now measured exactly, whereas previously it depended on predicting the begindifference of the next syllable, as well as issue #1725. So cleared syllables are tighter than before.

Breaks two new tests:
gabc-output/glyphs/clear.gabc: because of the changes to clearing described above
tex-output/bar-spacing-old/bar-spacing-old.tex: because the actual begindifference of a bar is used, whereas before it was 0

…en a syllable and the next, a hyphen is inserted. This used to be done in \GreSyllable, but now is done in the Lua pre_linebreak filter.

- Added some missing cases to \gre@calculate@eolshift
- Remove TeX code computing syllablefinalskip and code that depends on it
  * It is still used inside the new bar spacing algorithm, but for a different purpose
  * There is still a macro \gre@calculate@syllablefinalskip, but it actually computes minNotesDistance and minTextDistance
- Intra-line and end-of-line hyphens are now generated by the same code
  * One very minor side effect is that end-of-line hyphens should now be correctly kerned.
@rpspringuel rpspringuel added this to the 7.0.0 milestone Mar 18, 2026
@davidweichiang
Copy link
Copy Markdown
Contributor Author

davidweichiang commented Mar 18, 2026

Commit a48c861 moves hyphenation into Lua. This was the last thing that depended on syllablefinalskip, so syllablefinalskip can be deleted from the TeX code, and perhaps it would make sense to merge (so that @lbssousa can work on #1715), after 6.2.0 of course.

Some very minor benefits:

  • A couple of cases in \gre@calculate@eolshift appeared to be incorrect or missing; these are now fixed.
  • End-of-line hyphens used to not be kerned, but now they are.
  • When lyrics are hidden, the old code inserted extra stretch after hyphenated syllables. The new code does not. This affects one test:
    • tex-output/hidelyricsnotes/hidelyricsnotes.tex

Several more tests are broken.

If a hyphenated word comes before a bar, the space between them is now incorrect until bar spacing is moved into Lua. This happens in the following tests:

  • gabc-output/glyphs/custos_linebreak.gabc
  • tex-output/bar-spacing/bar-spacing-new.tex
  • tex-output/bar-spacing/lyric-centering.tex: only the lines printed for debugging are affected

Miniscule differences:

  • gabc-output/two-nabc.gabc
  • gabc-output/bugs/fix-1424.gabc
  • gabc-output/bugs/fix-804.gabc
  • gabc-output/bugs/fix-347.gabc
  • tex-output/illumination/illumination.tex
  • tex-output/bugs/fix-1110/fix-1110.tex
  • tex-output/bugs/fix-1319/fix-1319.tex
  • tex-output/snippet/snippet.tex
  • tex-output/hidelyricsnotes/text-only.tex

More than miniscule but not worrisome in my opinion:

  • tex-output/visibility-controls/visibility-controls-main.tex

@davidweichiang
Copy link
Copy Markdown
Contributor Author

This code defers some computations to the pre-linebreak filter, which can depend on user settings. If the user changes settings in the middle of the score (using [ev:...] or the like) then the change will affect the entire score, which may be surprising. In the tests, this only happens in bar-mora.gabc, which is not affected here. But in principle, are there settings that need to be settable in the middle of the score?

@rpspringuel

Comment thread doc/Command_Index_internal.tex Outdated
Comment thread doc/Command_Index_internal.tex Outdated
@rpspringuel
Copy link
Copy Markdown
Contributor

I'd definitely like to see some documentation on the new Lua code before merging. We've been very poor about documenting our Lua code thus far and I'd like to start changing that.

I'm thinking that our policy for new Lua code should be as follows:

Every new function (or non-trivial change to an existing function) gets a documentation comment in EmmyLua-style:

--- Calculates the area of a rectangle
---@param width number The width of the rectangle
---@param height number The height of the rectangle
---@return number area The computed area
local function area(width, height)
    return width * height
end

Do you think that's reasonable? The goal is to improve our Lua documentation over time without there being an overwhelming amount of work. (Though if someone gets ambitious and wants to document all current Lua functions, I won't stop them. That's how I really got into the project: creating the TeX documentation.)

@eschwab
Copy link
Copy Markdown
Contributor

eschwab commented Apr 20, 2026

The code documentation suggestion seems good. I started a discussion on the gregorio-devel mailing list in an attempt to standardize our lua style.

@davidweichiang
Copy link
Copy Markdown
Contributor Author

The last commit takes a first stab at adding the Lua comments.

@davidweichiang
Copy link
Copy Markdown
Contributor Author

davidweichiang commented Apr 27, 2026

To fix:

  • Position hyphen correctly in a rewritten syllable (visible in test fix-396.tex)
  • Improve positioning of clef changes without bars

Copy link
Copy Markdown
Contributor

@rpspringuel rpspringuel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The addition of the Lua comments makes it much easier to read through the new Lua file. Thank you.

Still need CHANGELOG entry. Based on what I saw, I don't think any UPGRADE entry is needed.

Comment thread tex/gregoriotex-main.tex Outdated
Comment thread tex/gregoriotex-syllable.lua Outdated
@davidweichiang
Copy link
Copy Markdown
Contributor Author

I believe merging #1743 first will make this one easier, as it will fix a few failing tests.

@davidweichiang
Copy link
Copy Markdown
Contributor Author

davidweichiang commented May 4, 2026

To fix:

  • When a hyphen is added, kerning is not happening when it should

@davidweichiang davidweichiang marked this pull request as draft May 6, 2026 01:32
…ng/ligaturing after syllable rewriting. This fixes the problem.

When two syllables are merged, the link for the first syllable spans both syllables, and the original link for the second syllable is lost.
@davidweichiang davidweichiang marked this pull request as ready for review May 9, 2026 20:10
@davidweichiang
Copy link
Copy Markdown
Contributor Author

The last outstanding issue with this is that there are several settings that are read at pre-linebreak time. If the user changes any of these mid-score, the change will affect the entire score, which is probably not what they want.

Currently, I believe it's just:

  • maximumspacewithoutdash
  • intersyllablespacestretchhyphen
  • \ifgre@showlyrics
  • \ifgre@rewritesyllables
    This list will grow as more code gets moved into Lua.

I think it will be better to save these settings per-syllable. Should it just save all of them, or are there any settings that are inconceivable that anyone would want to change them mid-score?

@rpspringuel
Copy link
Copy Markdown
Contributor

I would normally have said all those were score-wide settings, but as I think about it, I can conceive of situations where someone might want to change them mid-score, at least for a few syllables.

My question now is whether we should try to implement this as a defaults-exceptions scheme (where the changes inside a score create exceptions for some number of syllables, up to the rest of the score, but leave the original settings intact), or simply allow the user to change the settings mid-score and rely on them to change things back for the next score if that is what they want>

…ole syllable is bold or italic, the hyphen is too. This is the same behavior as the old code for intra-line syllables. But it is different from the old code for end-of-line syllables; the old code matched the hyphen's font to the last character's.
@davidweichiang
Copy link
Copy Markdown
Contributor Author

It looks like mid-score changes persist to the next score, so I think that should stay the same. It also looks like:

  • [ev:\gresetsyllablerewriting{off}] drops characters: Mid-score \gresetsyllablerewriting{off} loses characters #1756
  • <v>\gresetsyllablerewriting{off}</v> (or any other setting) inside text has no effect, because text is expanded inside a group
  • [ev:\grechangedim{maximumspacewithoutdash}{1cm}{scalable}]) takes effect starting from the following syllable, which I find mildly surprising but I guess it's ok:
    \gabcsnippet{a(g)a(g[ev:\grechangedim{maximumspacewithoutdash}{1cm}{scalable}])a(g)a(g)}
image

@davidweichiang davidweichiang linked an issue May 13, 2026 that may be closed by this pull request
@davidweichiang
Copy link
Copy Markdown
Contributor Author

The above commit was the last thing I wanted to fix, so I think this could be really ready now.

… algorithm)"

This reverts commit 4ac81fc, which was incorrect.

The problem is that when a hyphen is inserted, it does not move the end-of-syllable boundary. This is fine because adjust_syllablefinalskip does this already. However, if the next syllable is a bar syllable and the old bar spacing algorithm is used, there is no syllablefinalskip to adjust. The result is that the hyphen could protrude too far.

This can be fixed correctly, but causes other problems. It seems better to defer this to a proper fix of issue gregorio-project#1755.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mid-score \gresetsyllablerewriting{off} loses characters <clear> undoes wrong syllablefinalskip Syllable rewriting bugs

3 participants