feat: load the model once and keep it loaded

[kyteinsky]

No description provided.

[Copilot]

Pull Request Overview

This PR refactors the translation service from using a context manager pattern to a persistent model loading approach, improving performance by loading the model once at thread initialization rather than for each translation request.

Key changes:

• Introduced load_model() method to initialize tokenizer and translator as instance attributes
• Removed the translate_context context manager and its resource cleanup logic
• Modified translate() to use persistent self.tokenizer and self.translator instead of context manager resources

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
lib/main.py	Adds call to service.load_model() in the task fetch thread to initialize the model before processing tasks
lib/Service.py	Refactors from context manager pattern to persistent model loading with new load_model() method and updated translate() method to use instance attributes

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[oleksandr-nc]

this PR needs to be rebased after a fix for ROCM PR

[kyteinsky]

this PR needs to be rebased after a fix for ROCM PR

I was hoping merging this first would have solved this but probably not.
Will do.

[oleksandr-nc]

changes looks ok (but did not test this, hopes it works good)

[kyteinsky]

works on my machine (TM)
thanks for the review!