Add HTS label support infrastructure

[rokujyushi]

This pull request refactors and enhances the EnunuOnnx phonemizer and its supporting utilities in OpenUtau. The main changes include moving and reorganizing utility files, improving HTS note and phoneme feature calculations, and enhancing the data passed between rendering components for better extensibility and accuracy.

Refactoring and File Organization:

• Moved utility classes (HTSLabelFile.cs, Merlin.cs, Python.cs, Scaler.cs) from OpenUtau.Plugin.Builtin/EnunuOnnx/ to OpenUtau.Core/Util/ and updated namespaces and references to reflect this change. This improves code organization and reuse. [1] [2] [3] [4] [5]

HTS Note and Phoneme Feature Improvements:

• Enhanced the creation of HTSNote objects to include additional musical and timing information such as time signature, bar/beat position, slur/rest status, key, language, accent, and BPM. This provides richer context for downstream processing. [1] [2]
• Improved the calculation of distances to vowels for consonant phonemes in HTSPhoneme, providing more accurate context features for each phoneme.

Render Pipeline Enhancements:

• Updated RenderPhone and RenderPhrase to pass and track the note index for each phoneme, allowing more precise mapping between notes and their phonemes during rendering. [1] [2] [3] [4]

HTS Data Structure Updates:

• Replaced the standalone HTS.cs file with more modular and maintainable classes, and updated the way HTSNote and HTSPhoneme link to their parent phrase and each other, supporting richer relationships and easier traversal. [1] [2]

These changes collectively improve code maintainability, extensibility, and the accuracy of phoneme feature extraction, which is crucial for high-quality singing synthesis.

---

Refactoring and File Organization

• Moved utility files from OpenUtau.Plugin.Builtin/EnunuOnnx/ to OpenUtau.Core/Util/ and updated namespaces and references throughout the codebase. [1] [2] [3] [4] [5]

HTS Note and Phoneme Feature Improvements

• Enhanced HTSNote construction to include time signature, bar/beat position, slur/rest status, key, language, accent, and BPM for each note, improving context for phoneme processing. [1] [2]
• Improved vowel distance calculations for consonant phonemes in HTSPhoneme, increasing the accuracy of phoneme context features.

Render Pipeline Enhancements

• Passed note index information into RenderPhone and tracked note indexes in RenderPhrase for more accurate mapping between notes and phonemes. [1] [2] [3] [4]

HTS Data Structure Updates

• Updated HTS data structures to link notes and phonemes to their parent phrase and each other, and removed the old monolithic HTS.cs file in favor of a more modular approach. [1] [2]

[Copilot]

Pull request overview

This PR introduces new HTS label generation infrastructure and refactors existing ENUNU/nnmnkwii utilities into OpenUtau.Core to support richer HTS note/phoneme context features and better data flow through the render pipeline.

Changes:

• Added new HTS core model + context builder (OpenUtau.Core/Util/HTS.cs) and new HTS label renderer/phonemizer bases (OpenUtau.Core/Hts/*).
• Refactored nnmnkwii-related utility classes into OpenUtau.Core/Util/* and updated ENUNU ONNX phonemizer imports/usages accordingly.
• Extended render pipeline data to track per-phoneme noteIndex and added tests for HTS spec/pipeline behavior.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
OpenUtau.Test/Plugins/HtsLabelPhonemizerTest.cs	Adds an end-to-end-ish HTS label + frontend feature extraction test using a dummy phonemizer.
OpenUtau.Test/Core/Util/HtsSpecTests.cs	Adds spec-like unit tests for HTS note/phrase context fields and alignment helpers.
OpenUtau.Plugin.Builtin/EnunuOnnx/HTS.cs	Removes the old monolithic HTS implementation from the ENUNU plugin.
OpenUtau.Plugin.Builtin/EnunuOnnx/EnunuOnnxPhonemizer.cs	Updates to new HTS structures and adds richer HTS note context + phoneme vowel-distance features.
OpenUtau.Core/Util/Scaler.cs	Moves Scaler into OpenUtau.Core.Util namespace for broader reuse.
OpenUtau.Core/Util/Python.cs	Moves nnmnkwii python exception types into OpenUtau.Core.Util.nnmnkwii.python.
OpenUtau.Core/Util/Merlin.cs	Moves Merlin frontend implementation into OpenUtau.Core.Util.nnmnkwii.frontend.
OpenUtau.Core/Util/HTSLabelFile.cs	Moves HTS label IO into OpenUtau.Core.Util.nnmnkwii.io.hts.
OpenUtau.Core/Util/HTS.cs	Introduces new HTS note/phoneme/phrase classes and alignment/context-building helpers.
OpenUtau.Core/Render/RenderPhrase.cs	Adds noteIndex propagation from notes to phones for better note↔phoneme mapping.
OpenUtau.Core/Hts/HTSLabelRenderer.cs	Adds a new renderer base that can emit HTS labels from RenderPhrase.
OpenUtau.Core/Hts/HTSLabelPhonemizer.cs	Adds a new phonemizer base that can generate HTS labels and align timings back into OpenUtau phonemes.

Comments suppressed due to low confidence (2)

OpenUtau.Core/Util/HTS.cs:727

• Backward measure contexts likely have the same off-by-one issue: measureIndexBackward is set to group.Count - noteIndex (1-based) and measurePercentBackward uses (totalNotesInMeasure - 1) as denominator. The spec tests added in this PR expect 0-based backward indexes and percentages like 66/33/0 for 3 notes (denominator = totalNotesInMeasure).

                for (int noteIndex = group.Count - 1; noteIndex >= 0; --noteIndex) {
                    var note = group[noteIndex];
                    int backwardIndex = group.Count - noteIndex;
                    note.measureIndexBackward = backwardIndex;
                    note.measureMsBackward = (int)Math.Round(accMsB / 100.0);
                    note.measureTickBackward = ticksPer96th > 0 ? (int)Math.Round((double)accTicksB / ticksPer96th) : 0;
                    note.measurePercentBackward = totalNotesInMeasure > 1 ? ((backwardIndex - 1) * 100) / (totalNotesInMeasure - 1) : 0;

OpenUtau.Core/Hts/HTSLabelPhonemizer.cs:74

• g2p is loaded before LoadDict, but LoadG2p adds entries from phoneDict—which is still empty at that point. Also LoadG2p is called with singer.Location instead of the computed rootPath, so it may read the wrong enunux.yaml. Consider loading the dictionary first (as EnunuOnnxPhonemizer does) and passing rootPath into LoadG2p.

            //Load g2p from enunux.yaml
            //g2p dict should be load after enunu dict
            try {
                g2p = LoadG2p(singer.Location);
            } catch (Exception e) {
                Log.Error(e, "failed to load g2p dictionary");
                return;
            }
            //Load Dictionary
            var enunuDictPath = Path.Join(rootPath, tablePath);
            try {
                LoadDict(Path.Join(rootPath, tablePath), singer.TextFileEncoding);

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

OpenUtau.Plugin.Builtin/EnunuOnnx/EnunuOnnxPhonemizer.cs:563

• When building the HTS note list, htsNotes[i] no longer has index, indexBackwards, sentenceDurMs, or sentenceDurTicks populated (these assignments were removed). HTSNote.e() uses these fields for multiple contexts, so the dumped labels will contain incorrect/"xx"/potentially negative values (e.g., backward time). Restore initialization of these per-note fields (and consider whether padding notes should be excluded from totals).

            var htsPhrase = new HTSPhrase(htsNotes.ToArray());
            htsPhrase.totalNotes = htsNotes.Count;
            htsPhrase.totalPhonemes = htsPhonemes.Count;
            //make neighborhood links between htsNotes and between htsPhonemes
            foreach (int i in Enumerable.Range(0, htsNotes.Count)) {
                htsNotes[i].parent = htsPhrase;
                if (i > 0) {
                    htsNotes[i].prev = htsNotes[i - 1];
                    htsNotes[i - 1].next = htsNotes[i];
                }
            }