Add dense score retrieval for certain ScoreAccessors by SimBe195 · Pull Request #223 · rwth-i6/rasr

SimBe195 · 2026-05-15T10:23:57Z

Some ScoreAccessors (e.g. VectorScoreAccessor) represent a dense span of score values over the vocabulary that don't depend on the transition type. For these ScoreAccessor classes we can implement an optimized function to retrieve the full dense score span directly and thus fetch all scores at once. This avoids repeated virtual scoreAccessor->getScore(transitionType, labelIndex) function calls in the search algorithms for every possible extension. In the future it can also be used to introduce local pruning over the dense score span when it's available.

Originally I modeled the DenseScoreSpan just as a std::span<Score const> but that doesn't support scaling and interpolation as it is used in CombineLabelScorer, PriorLabelScorer and ScaledLabelScorer. So now it is instead modeled as a struct containing multiple terms each of which consist of a span + a scale.

In a test segment with a speech LLM and lexiconfree labelsync search, the dense score accessors reduce the search time from 13.39s to 12.85s (RTF 0.336 to 0.323).

…essors

larissakl · 2026-05-22T06:52:15Z

+    size_t size() const {
+        return terms.empty() ? 0ul : terms.front().scores.size();
+    }


I think this might be confusing because one would usually assume to get the size of the vector terms when calling size() for this struct, not size of the scores in terms. Maybe we could rename the function to have it more explicit?

I don't mind either way, but it is consistent in the view that it returns the maximum index + 1 for the array accessor below. In that sense it is like a vector.

larissakl · 2026-05-22T06:58:06Z

+
+        std::vector<DenseScoreTerm> denseScoreTerms(denseScores->terms.begin(), denseScores->terms.end());
+        for (auto& term : denseScoreTerms) {
+            term.scale *= scale_;


Why *= and not =?

Because the terms might come from a combined label scorer that does scaling differently for each term. Then the scaled label scorer here should multiply its own scale and not override the scales of the combined label scorer.

larissakl · 2026-05-22T07:11:27Z

+                        if (denseScores and tokenIdx < denseScores->size()) {
+                            extScore += (*denseScores)[tokenIdx];
+                        }
+                        else {
+                            extScore += (*scoreAccessor)->getScore(transitionType, tokenIdx);
+                        }


Why do you use an if-else block here and not the same syntax as below?

larissakl · 2026-05-22T07:25:45Z

+            else if (denseScores->size() != denseSize) {
+                return std::nullopt;
+            }
+            denseScoreTerms.insert(denseScoreTerms.end(), denseScores->terms.begin(), denseScores->terms.end());


I don't get the implementation for this ScoreAccessor. Why do you just collect the dense score terms of all subAccessors in one vector? Don't we need to sum them up?

curufinwe · 2026-05-26T09:38:29Z

+    size_t size() const {
+        return terms.empty() ? 0ul : terms.front().scores.size();
+    }


I don't mind either way, but it is consistent in the view that it returns the maximum index + 1 for the array accessor below. In that sense it is like a vector.

curufinwe · 2026-05-26T09:39:54Z

+    std::vector<DenseScoreTerm> terms;
+
+    DenseScoreSpan(std::vector<DenseScoreTerm>&& terms)
+            : terms(std::move(terms)) {}


Maybe we could add an assertion here to check that all terms are of the same length.

curufinwe · 2026-05-26T09:47:31Z

+
+        std::vector<DenseScoreTerm> denseScoreTerms(denseScores->terms.begin(), denseScores->terms.end());
+        for (auto& term : denseScoreTerms) {
+            term.scale *= scale_;


Because the terms might come from a combined label scorer that does scaling differently for each term. Then the scaled label scorer here should multiply its own scale and not override the scales of the combined label scorer.

curufinwe · 2026-05-26T09:48:38Z

+        for (auto& term : denseScoreTerms) {
+            term.scale *= scale_;
+        }
+        return DenseScoreSpan(std::move(denseScoreTerms));


Why do we need to create a new DenseScoreSpan here instead of returning the modified one?

curufinwe · 2026-05-26T09:49:26Z

+            return std::nullopt;
+        }
+
+	std::vector<Nn::DenseScoreTerm> denseScoreTerms(denseScores->terms.begin(), denseScores->terms.end());


Suggested change

std::vector<Nn::DenseScoreTerm> denseScoreTerms(denseScores->terms.begin(), denseScores->terms.end());

std::vector<Nn::DenseScoreTerm> denseScoreTerms(denseScores->terms.begin(), denseScores->terms.end());

But actually: why do we need a copy? We could also just update denseScores, no?

curufinwe · 2026-05-26T09:50:48Z

+    Core::Ref<ScoreAccessor>                scoreAccessor_;
+    const bool                              negateInput_;
+    std::shared_ptr<Nn::Prior<Score>>       prior_;


why this whitespace change?

curufinwe · 2026-05-26T09:53:51Z

+            denseScoreTerms.push_back(Nn::DenseScoreTerm{denseScores->size() > 0ul ? std::span<Score const>(&prior_->at(0), prior_->size()) : std::span<Score const>(), prior_->scale()});
+        }
+
+        return Nn::DenseScoreSpan(std::move(denseScoreTerms));


Why create a new DenseScoreSpan instead of returning the modified one?

curufinwe · 2026-05-26T09:59:50Z

                auto const& scoreAccessor = scoreAccessors[hypIndexToContextIndexMap_[ext.baseHypIndex]];

                if (scoreAccessor) {
                    ext.score += (*scoreAccessor)->getScore(ext.transitionType, ext.nextToken);


Shouldn't we also use the dense scores here?

SimBe195 added 4 commits May 13, 2026 15:04

Implement option for returning dense score spans in certain score acc…

b73ed6b

…essors

Use the dense score view also in TreeTimesyncBeamSearch

24af9a9

Merge branch 'multi_scorer_labelsync' into dense_score_accessors

f2690c2

Merge branch 'multi_scorer_labelsync' into dense_score_accessors

3cf80ea

SimBe195 requested review from curufinwe, hannah220 and larissakl May 15, 2026 10:23

larissakl reviewed May 22, 2026

View reviewed changes

curufinwe requested changes May 26, 2026

View reviewed changes

	std::vector<Nn::DenseScoreTerm> denseScoreTerms(denseScores->terms.begin(), denseScores->terms.end());
	std::vector<Nn::DenseScoreTerm> denseScoreTerms(denseScores->terms.begin(), denseScores->terms.end());

Conversation

SimBe195 commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SimBe195 commented May 15, 2026 •

edited

Loading