RE: Automating Multi-Lingual and Multi-Speaker Closed-Captioning and Transcripting Workflow with srt2vtt
I found one that seemed to work quite well, and another one that was staring us in the face. It turns out that YouTube itself generates word-by-word aligned captions!
Wow! I didn't know that!
However, the interesting thing with the YouTube word aligned captions is that they still seem slightly off, perhaps to make them more "fluid" while following the captions on screen. Another interesting issue I found was that while I could download these precise timings in a WebVTT file, if I uploaded the exact same file back to YouTube, it no longer worked correctly!
The story gets murkier by the minute. Besides of YouTube timing issue, are you sure you're not complicating it yourself? ;)
My intended workflow which has to be tested:
- Record multitrack audio
- get Speech-To-Text with time codes for each track separately
- Run an aligner like aeneas or Prosodylab to increase the accuracy i.e. get time stamps for each word
- Merge those files with srt2vtt
- Edit the resulting file by removing redundant words
- Now use the punctuator if you hadn't already manually - the preffered way
- Cut the audio against the the time stamps that are left with videogrep
- Realign the timings using one of the aligners above.
- Create transcript with srt2vtt
So this should be it or you might point out what I'm missing?
It's important to know the difference between Speech-To-Text and aligning words. Of which the latter should be much easier because you're just guessing which word will be next instead of guessing what word is being spoken or if it is just noise.
Also note that when speaker IDs are inserted into the text that we should not try to align those since they are actually not there.
did you watch the sample YouTube video in my post? Try changing between "English" captions and "English (auto-generated)" to see the difference in display and timings (both of which were excellent for most use cases I would envision, at least with this particular audio sample).
YouTube's captioning before speaker IDs are added, Gentile simply skips over them. But of course, as I explained, using word-by-word captioning may be mostly unnecessary if all audio tracks are simultaneously "corrected" as follows:
1a. Load all tracks into a multitrack audio editor, cut noise/"utterances" over all clips simultaneously, export individual edited tracks
...
Sure I agree that multitrack cleaning is the way to go but SteemPowerPics didn't know how even when I showed it should be possible...
I'm not sure about the speaker IDs, if the speakerIDs are like words/names are in the transcript how can you skip them? With [something] like square brackets?
Kind of missed that part but replace step 3 with anything which works according to your needs right?