ROCm NeuTTS backend throws exception on startup

[rabits]

LocalAI version:
Docker localai/localai:latest-gpu-hipblas, v3.10.1 (923ebbb)

Environment, CPU architecture, OS, and Version:

• Linux workstation-1 6.14.0-35-generic #35~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Oct 14 13:55:17 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
• 2x Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, 128GB RAM
• Radeon RX 9060 XT 16GB

Describe the bug
When running: see it's failing during executing run.sh by not finding the module: ModuleNotFoundError: No module named 'neutts' - tried the same from the container (just docker exec bash and then get to backend/rocm-neutts and ./run.sh) - throws the same error.

To Reproduce
Install backend rocm-neutts, download & configured model neutts-air-Q4-0.gguf:

name: neutts-air-Q4-0.gguf
backend: neutts
parameters:
  model: neutts-air-Q4-0.gguf
description: Imported from huggingface://neuphonic/neutts-air-q4-gguf/neutts-air-Q4-0.gguf
known_usecases:
  - tts
tts:
  audio_path: /backends/blabla.wav
options:
  - 'ref_text:blablabla'

And try to run TTS.

Expected behavior
To work when running.

Logs

Feb 05 03:21:19 DEBUG LocalAI TTS Request received model="neutts-air-Q4-0.gguf" caller={caller.file="/build/core/http/endpoints/localai/tts.go"  caller.L=39 }
Feb 05 03:21:19 INFO  BackendLoader starting modelID="neutts-air-Q4-0.gguf" backend="neutts" model="neutts-air-Q4-0.gguf" caller={caller.file="/build/pkg/model/initializers.go"  caller.L=159 }
Feb 05 03:21:19 DEBUG Loading model in memory from file file="/models/neutts-air-Q4-0.gguf" caller={caller.file="/build/pkg/model/loader.go"  caller.L=216 }
Feb 05 03:21:19 DEBUG Loading Model with gRPC modelID="neutts-air-Q4-0.gguf" file="/models/neutts-air-Q4-0.gguf" backend="neutts" options={neutts neutts-air-Q4-0.gguf neutts-air-Q4-0.gguf {{}} 0xc000399208 map[] 20 2 false} caller={caller.file="/build/pkg/model/initializers.go"  caller.L=53 }
Feb 05 03:21:19 DEBUG Loading external backend uri="/backends/rocm-neutts/run.sh" caller={caller.file="/build/pkg/model/initializers.go"  caller.L=77 }
Feb 05 03:21:19 DEBUG external backend is file file=&{run.sh 192 493 {0 63904774957 0x4ed42c0} {2304 26874180 1 33261 0 0 0 0 192 4096 8 {1769178157 0} {1769178157 0} {1770259131 552244574} [0 0 0]}} caller={caller.file="/build/pkg/model/initializers.go"  caller.L=80 }
Feb 05 03:21:19 DEBUG Loading GRPC Process process="/backends/rocm-neutts/run.sh" caller={caller.file="/build/pkg/model/process.go"  caller.L=108 }
Feb 05 03:21:19 DEBUG GRPC Service will be running id="neutts-air-Q4-0.gguf" address="127.0.0.1:37139" caller={caller.file="/build/pkg/model/process.go"  caller.L=110 }
Feb 05 03:21:19 DEBUG GRPC Service state dir dir="/tmp/go-processmanager1062829493" caller={caller.file="/build/pkg/model/process.go"  caller.L=134 }
Feb 05 03:21:19 DEBUG GRPC Service Started caller={caller.file="/build/pkg/model/initializers.go"  caller.L=92 }
Feb 05 03:21:19 DEBUG Wait for the service to start up caller={caller.file="/build/pkg/model/initializers.go"  caller.L=105 }
Feb 05 03:21:19 DEBUG Options options=ContextSize:32768  Seed:1913743869  NBatch:512  MMap:true  NGPULayers:25  Threads:12  AudioPath:"/backends/<REDACTED>.wav"  FlashAttention:"auto"  Options:"ref_text:<REDACTED>"  Options:"use_jinja:true" caller={caller.file="/build/pkg/model/initializers.go"  caller.L=106 }
Feb 05 03:21:19 DEBUG GRPC stdout id="neutts-air-Q4-0.gguf-127.0.0.1:37139" line="Initializing libbackend for rocm-neutts" caller={caller.file="/build/pkg/model/process.go"  caller.L=158 }
Feb 05 03:21:19 DEBUG GRPC stdout id="neutts-air-Q4-0.gguf-127.0.0.1:37139" line="Using portable Python" caller={caller.file="/build/pkg/model/process.go"  caller.L=158 }
Feb 05 03:21:20 DEBUG GRPC stdout id="neutts-air-Q4-0.gguf-127.0.0.1:37139" line="Added /backends/rocm-neutts/lib to LD_LIBRARY_PATH for GPU libraries" caller={caller.file="/build/pkg/model/process.go"  caller.L=158 }
Feb 05 03:21:23 DEBUG GRPC stderr id="neutts-air-Q4-0.gguf-127.0.0.1:37139" line="Traceback (most recent call last):" caller={caller.file="/build/pkg/model/process.go"  caller.L=149 }
Feb 05 03:21:23 DEBUG GRPC stderr id="neutts-air-Q4-0.gguf-127.0.0.1:37139" line="  File \"/backends/rocm-neutts/backend.py\", line 14, in <module>" caller={caller.file="/build/pkg/model/process.go"  caller.L=149 }
Feb 05 03:21:23 DEBUG GRPC stderr id="neutts-air-Q4-0.gguf-127.0.0.1:37139" line="    from neuttsair.neutts import NeuTTSAir" caller={caller.file="/build/pkg/model/process.go"  caller.L=149 }
Feb 05 03:21:23 DEBUG GRPC stderr id="neutts-air-Q4-0.gguf-127.0.0.1:37139" line="  File \"/backends/rocm-neutts/neuttsair/__init__.py\", line 1, in <module>" caller={caller.file="/build/pkg/model/process.go"  caller.L=149 }
Feb 05 03:21:23 DEBUG GRPC stderr id="neutts-air-Q4-0.gguf-127.0.0.1:37139" line="    from .neutts import NeuTTSAir" caller={caller.file="/build/pkg/model/process.go"  caller.L=149 }
Feb 05 03:21:23 DEBUG GRPC stderr id="neutts-air-Q4-0.gguf-127.0.0.1:37139" line="  File \"/backends/rocm-neutts/neuttsair/neutts.py\", line 1, in<module>" caller={caller.file="/build/pkg/model/process.go"  caller.L=149 }
Feb 05 03:21:23 DEBUG GRPC stderr id="neutts-air-Q4-0.gguf-127.0.0.1:37139" line="    from neutts.neutts import NeuTTS" caller={caller.file="/build/pkg/model/process.go"  caller.L=149 }
Feb 05 03:21:23 DEBUG GRPC stderr id="neutts-air-Q4-0.gguf-127.0.0.1:37139" line="ModuleNotFoundError: No module named 'neutts'" caller={caller.file="/build/pkg/model/process.go"  caller.L=149 }
Feb 05 03:21:57 ERROR failed starting/connecting to the gRPC service error=rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:37139: connect: connection refused" caller={caller.file="/build/pkg/model/initializers.go"  caller.L=118}

From docker exec shell:

root@d0a692fa02e4:/backends/rocm-neutts# ./run.sh
Initializing libbackend for rocm-neutts
Using portable Python
Added /backends/rocm-neutts/lib to LD_LIBRARY_PATH for GPU libraries
Traceback (most recent call last):
  File "/backends/rocm-neutts/backend.py", line 14, in <module>
    from neuttsair.neutts import NeuTTSAir
  File "/backends/rocm-neutts/neuttsair/__init__.py", line 1, in <module>
    from .neutts import NeuTTSAir
  File "/backends/rocm-neutts/neuttsair/neutts.py", line 1, in <module>
    from neutts.neutts import NeuTTS
ModuleNotFoundError: No module named 'neutts'

Additional context

[rabits]

Seems like it was refactored and tried to keep the backward compatibility, but used global scope for that, which is not working anymore: neuphonic/neutts#87 .

I think the easy fix would be to specify which revision is actually cloned during install.sh - it's a good practice to keep the system resilient to the uncontrolled changes. I've checked by manually checkout and copy with the next commands:

$ git clone https://github.com/neuphonic/neutts-air neutts-air
$ git -C neutts-air checkout 1737487debe5b40a0bb97875edce8c66b391722b
$ cp -rfv neutts-air/neuttsair ./

Unfortunately it did not like my local gguf model file, so I had to set to model: neuphonic/neutts-air as in PR example:

Feb 05 22:55:15 DEBUG GRPC stderr id="neutts-air-Q4-0.gguf-127.0.0.1:45631" line="CUDA is available" caller={caller.file="/build/pkg/model/process.go"  caller.L=149 }
Feb 05 22:55:15 DEBUG GRPC stderr id="neutts-air-Q4-0.gguf-127.0.0.1:45631" line="Preparing models, please wait" caller={caller.file="/build/pkg/model/process.go"  caller.L=149 }
Feb 05 22:55:16 DEBUG GRPC stdout id="neutts-air-Q4-0.gguf-127.0.0.1:45631" line="Loading phonemizer..." caller={caller.file="/build/pkg/model/process.go"  caller.L=158 }
Feb 05 22:55:16 DEBUG GRPC stdout id="neutts-air-Q4-0.gguf-127.0.0.1:45631" line="Loading backbone from: neutts-air-Q4-0.gguf on cuda ..." caller={caller.file="/build/pkg/model/process.go"  caller.L=158 }
Feb 05 22:55:16 DEBUG GRPC stdout id="neutts-air-Q4-0.gguf-127.0.0.1:45631" line="Received termination signal. Shutting down..." caller={caller.file="/build/pkg/model/process.go"  caller.L=158 }
Feb 05 22:55:19 ERROR Failed to load model modelID="neutts-air-Q4-0.gguf" error=failed to load model with internal loader: could not load model (no success): Unexpected err=NotImplementedError('Access to repositories lists is not implemented.'), type(err)=<class 'NotImplementedError'> backend="neutts" caller={caller.file="/build/pkg/model/initializers.go"  caller.L=179 }

Unfortunately it doesn't really support multilang, so I'm dropping it...