Wrong implementation of carry_initial_prompt

[alubbe]

When transcribing long form content, it is extremely helpful to give whisper information on how to spell names/terms (people, places, companies, products, jargon, etc.). This can be done for the beginning of the audio file using the 'initial prompt', but since this prompt is not persisted, this information is quickly lost.

OpenAI recently added a carry_initial_prompt option and huggingface's transformers have a prompt_condition_type=‘all-segments’ option - both of which try to ensure that the 'initial prompt' is always in the context.

The way I believe they do this is by saying that in each 30s segment's prompt, they prepend the initial prompt into the first 224 tokens of context, which are usually just the most recent 224 tokens of previous transcriptions (and as always, leaving at least 224 tokens for the actual transcription).

OpenAI's python implementation is very simple: https://github.com/openai/whisper/pull/2343/files

My hope is that it's also simple to implement this feature in this code base, like OpenAI and Huggingface transformers have done.

However, I

1. don't understand C/C++
2. don't understand the technical details of Whisper, Transformers, GPU code, etc.
3. the whisper.cpp file is too large even for Claude 3.5 Sonnet's and OpenAI's context windows, so they can't help

So here's a PR, mostly generated by LLMs seeing subsections of the codebase. The code itself doesn't work at all, and is probably completely invalid and bad.

But I hope this PR serves as a suggestion of how to port this simple OpenAI implementation of carry_initial_prompt over, and maybe even serves as motivation to finish this PR / build your own commit.

Alternatively, if you could point me in the right direction / write some pseudo code of what would have to happen, I'll be happy to attempt writing it!

And in any case, happy New Year!

[alubbe]

@ggerganov happy New Year! I hope you had a great start to the year so far - is there any chance you could give me some pointers on how this could be implemented? I'd love to work on it, and it shouldn't be too complicated, but I will need some help

[alubbe]

Closing in favor of #3395