[Proposal] Adding an option to enable/disable X-TIMESTAMP-MAP in WebVTT

[ozt860071]

Please prefix your issue with one of the following: [BUG], [PROPOSAL], [QUESTION].

CCExtractor version (using the --version parameter preferably) : X.X

In raising this issue, I confirm the following (please check boxes, eg [X] - and delete unchecked ones):

• I have read and understood the contributors guide.
• I have checked that the bug-fix I am reporting can be replicated, or that the feature I am suggesting isn't already present.
• I have checked that the issue I'm posting isn't already reported.
• I have checked that the issue I'm porting isn't already solved and no duplicates exist in closed issues and in opened issues
• I have checked the pull requests tab for existing solutions/implementations to my issue/suggestion.
• I have used the latest available version of CCExtractor to verify this issue exists.

My familiarity with the project is as follows (check one, eg [X] - and delete unchecked ones):

• I have never used CCExtractor.
• I have used CCExtractor just a couple of times.
• I absolutely love CCExtractor, but have not contributed previously.
• I am an active contributor to CCExtractor.

Necessary information

• Is this a regression (did it work before)? [x] NO | [ ] YES - please specify the last known working version
• What platform did you use? [ ] Windows - [ ] Linux - [x] Mac
• What were the used arguments? -out=webvtt

Video links (replace text below with your links)

Additional information
Currently when WebVTT is specified as output, X-TIMESTAMP-MAP is automatically added. And if no subtitle text is found in the input, it generates an empty WebVTT, with just one line "WEBVTT" in the file.

There are potentially two issues with this implementation.
(a) X-TIMESTAMP-MAP is not a WebVTT requirement, but Apple HLS requirement, this may cause a problem if it is used in non-HLS environment. It should be an option to turn it off.
(b) On the other hand, Apple HLS spec says "an X-TIMESTAMP-MAP metadata header SHOULD be added to each WebVTT header", which includes the empty ones. The spec also says "If a WebVTT segment does not have the X-TIMESTAMP-MAP, the client MUST assume that the WebVTT cue time of 0 maps to an MPEG-2 timestamp of 0.". But unfortunately, not all clients follow this rule. It is safer to add X-TIMESTAMP-MAP in all VTT files, if it is enabled.

So, my proposal is to add an option for WebVTT, whether X-TIMESTAMP-MAP is enabled or disabled. And if it is enabled, it will be added in every VTT file it generates, including one with no caption cues.

(cf. "HTTP Live Streaming": https://tools.ietf.org/html/rfc8216 section 3.5)

[NilsIrl]

What happens if X-TIMESTAMP-MAP is disabled and the WebVTT cue time 0 doesn't map to an MPEG-2 timestamp of 0?

[NilsIrl]

What would be the reason to not have the X-TIMESTAMP-MAP?

[NilsIrl]

Wouldn't it be better to have the X-TIMESTAMP-MAP header present all the time?

[ozt860071]

You’ll lose caption, and in worst case, video freeze.

…

On Dec 4, 2019, at 3:34 PM, Nils ANDRÉ-CHANG ***@***.***> wrote: What happens if X-TIMESTAMP-MAP is disabled and the WebVTT cue time 0 doesn't map to an MPEG-2 timestamp of 0? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1127?email_source=notifications&email_token=AB4ZYEQBI7FXHMYUFSFJIE3QXA5B3A5CNFSM4JQUKKPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF64S7I#issuecomment-561891709>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4ZYERROT4I2YN2DRAMEYTQXA5B3ANCNFSM4JQUKKPA>.

[ozt860071]

Since it is HLS requirement, this may affect negatively in non-HLS use cases. I hope not. I don’t mind having it all the time, though.

…

On Dec 4, 2019, at 3:35 PM, Nils ANDRÉ-CHANG ***@***.***> wrote: What would be the reason to not have the X-TIMESTAMP-MAP? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1127?email_source=notifications&email_token=AB4ZYERUXAUFA2NTLMFRT43QXA5FLA5CNFSM4JQUKKPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF64VVY#issuecomment-561892055>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4ZYERROQQFDJSCDSCW4I3QXA5FLANCNFSM4JQUKKPA>.

[ozt860071]

I am fine with this. But I don’t know how it affects in other use cases where this is not required. Thanks

…

On Dec 4, 2019, at 3:40 PM, Nils ANDRÉ-CHANG ***@***.***> wrote: Wouldn't it be better to have the X-TIMESTAMP-MAP header present all the time? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1127?email_source=notifications&email_token=AB4ZYEXAGJ4VZMFAJXCOZELQXA5V7A5CNFSM4JQUKKPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF65C2A#issuecomment-561893736>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4ZYEXCUV3XZKLDLRY7BXTQXA5V7ANCNFSM4JQUKKPA>.

[NilsIrl]

@cfsmp3 What should I do? I feel like the best here is to always use X-TIMESTAMP-MAP.

[NilsIrl]

There is also the general problem that ccextractor creates files even when it can't find anything making this problem more general and not limited to .vtt. @cfsmp3

[ozt860071]

But for HLS at least, I need an output even if no caption text is contained in the .vtt file. The client needs the get the timing info from X-TIMESTAMP-MAP for synchronization. Thanks

…

On Dec 5, 2019, at 12:09 PM, Nils ANDRÉ-CHANG ***@***.***> wrote: There is also the general problem that ccextractor creates files even when it can't find anything making this problem more general and not limited to .vtt. @cfsmp3 <https://github.com/cfsmp3> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1127?email_source=notifications&email_token=AB4ZYEUVS7QIUBJILRT5MBLQXFNZBA5CNFSM4JQUKKPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGB7OQQ#issuecomment-562296642>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4ZYERHWKLVUU5W2PXJVT3QXFNZBANCNFSM4JQUKKPA>.

[NilsIrl]

The client needs the get the timing info from X-TIMESTAMP-MAP for synchronization.

why would you need to synchronize if you don't have subtitles?

[cfsmp3]

The client needs the get the timing info from X-TIMESTAMP-MAP for synchronization.

why would you need to synchronize if you don't have subtitles?

It's part of the HLS specification. If there are subtitles in WebVTT:

Subtitles

5. Subtitle requirements
   5.1. Subtitles MAY be provided.
   5.2. Subtitles MUST be WebVTT (according to the HLS specification) or IMSC1 in fMP4.
   5.3. WebVTT subtitles MUST be in text files, with an X-TIMESTAMP-MAP according to the HLS specification.

References:
https://developer.apple.com/documentation/http_live_streaming/hls_authoring_specification_for_apple_devices
https://tools.ietf.org/html/rfc8216#page-13

Note that Apple there says MUST (in 5.2) but the RPC8216 says SHOULD.
But still, if we generate a .vtt file, even if it has no subtitles, it should have the required header. Otherwise players may refuse to play anything at all.

[NilsIrl]

The thing is that you can't synchronize between 2 things when you're missing 1 of the 2, in this case subtitles.

[cfsmp3]

An empty subtitle file is not the same as no subtitle file. If there's a subtitle file even if it has no actual subtitles inside you need to be able to know where it starts relative to the video.

In any case if the specs say that's the way it is then that's what we have to implement, unless you want to get in touch (you can) with the RFC8216 writers and discuss :-)

Otherwise we're just not standard compliant and that's not a good thing to be. In fact, if you read the RFC you will see that in a bunch of places it says that clients MUST refuse to play media that is not compliant. Not that they should do their best to play it anyway, but the opposite.

And well, there's a good reason for that: To force all programs generating media to follow the specs.