Skip to content

Add model caching for LabelScorer creation#222

Open
SimBe195 wants to merge 9 commits into
masterfrom
cached_label_scorer_onnx_models
Open

Add model caching for LabelScorer creation#222
SimBe195 wants to merge 9 commits into
masterfrom
cached_label_scorer_onnx_models

Conversation

@SimBe195
Copy link
Copy Markdown
Collaborator

For the Encoder class we have a EncoderModelCache which can be used to cache associated models (e.g. ONNX models for the OnnxEncoder) to avoid loading them multiple times when creating multiple instances. This PR adds a similar mechanism for LabelScorer creation. Since some LabelScorers use multiple models (e.g. StatefulOnnxLabelScorer with the initializer-, updater- and scorer-model), the cache is keyed by the corresponding config selection. The cache class has been unified with the EncoderModelCache into a common class living in src/Nn/LabelScorer/ModelCache.hh.

Comment thread src/Nn/LabelScorer/StatefulOnnxLabelScorer.cc Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants