The pronunciation of a Chinese sentence

[mx305]

Checks

• This template is only for bug reports, usage problems go with 'Help Wanted'.
• I have thoroughly reviewed the project documentation but couldn't find information to solve my problem.
• I have searched for existing issues, including closed ones, and couldn't find a solution.
• I am using English to submit this issue to facilitate community communication.

Environment Details

ubuntu 22.04, Python 3.10, torch 2.4.0+cu124

Steps to Reproduce

text：多次与您沟通，一直未处理，您服务真的太差。
ref audio：默认的太乙真人。
question：It hasn’t been processed all this time, and the third character 未 is pronounced as wéi (the second tone). Tried to make modifications using the polyphonic character method shared by someone before, i.e., marking the character 未 as <wei4 未>, or change the character 未 to other homophones such as 卫, but the result remained unchanged.
Thanks.

✔️ Expected Behavior

如果遇到这类无法改声调或者多音字问题，如何解决呢？

❌ Actual Behavior

No response

[SWivid]

测了一下确实有这个问题。这种情况是参考音频的说话人带口音导致，目前的模型结构没有对音色和口音做解耦。可以通过换个参考音频解决。
Just confirmed that this issue does exist, which is caused by the accent of speaker. The current model structure does not perform disentanglement of timbre and accent, so you need to change the reference audio.

[mx305]

Thanks. The result was the same even after trying several reference audio.
Why did the replacement of homophonic characters also fail to resolve the issue? Any other solutions?

[SWivid]

@mx305 could use 多次与您沟通，一直未楚理，您服务真的太差。

我重新看了下，这个应该是jieba+pypinyin转多音字的时候的问题。猜测是处被认作第四声了，而按照中文连字逻辑，两个四声在一起的时候第一个可能会变调阳平。

marking the character 未 as <wei4 未>

直接修改指定最终的拼音序列final_text_list才行

F5-TTS/src/f5_tts/infer/utils_infer.py

Line 477 in 9ae46c8

final_text_list = convert_char_to_pinyin(text_list)

[mx305]

解决了jieba+pypinyin，输出都是'wei4', ' ', 'chu3', ' ', 'li3'，但同一个参考音频和同一句话生成多次，有时候还是不对，念wei3。
另外又发现一个"我不知道您什么意思。"。这个思对应的是si，结果直接不念了？是不支持轻声？
麻烦了！

[SWivid]

需要可复现流程：
ref_audio ref_text gen_text，最好有bad case对应的seed来确保我们能复现出来这个问题好分析

[mx305]

ref_audio/ref_text：(太乙真人)对，这就是我，万人敬仰的太乙真人

1. gen_text：多次与您沟通，一直未楚理。
   结果：生成10次有一半是念wei3。
2. gen_text：我不知道您什么意思。
   结果：“思”都没有输出。

关于seed：没有发现用到，也可能没留意到，如果是需要指定搞定，麻烦您指导下。
关于代码：使用最新版本，方法是infer_batch_process streaming，参数调整下面几个，其他都是默认的。
nfe_step=16
streaming=True

[SWivid]

gen_text：我不知道您什么意思。

可以用我不知道您什么意斯。hack一下

感觉主要原因还是这个音色本身就飘

[mx305]

“思”这个可以，试了下几十个比较固定。谢谢！
这个“未”帮忙看看。感觉掰不过来了。有没有其他参数可以调一下。

[SWivid]

固定下seed吧 如果是五五开的情况
类似这样加一行到你的脚本里

F5-TTS/src/f5_tts/infer/infer_gradio.py

Line 153 in c279a2b

torch.manual_seed(seed)