busybee fluffy rambles

It started out with the best of intentions. People who do this think they’re helping with accessibility by “adding captions.” Unfortunately, the nature of these captions actually hurts accessibility, and interferes with comprehension of the video for many people. It ends up grabbing the viewer’s attention and disrupting the ability to even read the words, as they’re constantly jumping around. Some people actually have worse difficulties comprehending what’s being spoken if there’s words on the screen, or have great difficulty reading the words if they’re animated.

Open captions also cannot be corrected if there are misspellings (such as “you’re” vs “your”) or incorrect word choices, as well as timing issues; a very common thing that occurs with various auto-captioning systems is that they’ll split up phrases awkwardly and end up having a portion of a sentence just flash on the screen for like a quarter of a second, which isn’t quite enough time to actually read it. Closed captions, on the other hand, can be corrected later, as the .srt file can simply be reuploaded, and some video platforms (such as YouTube) even let you edit them in-place online.

EDIT: I also forgot to mention that, relatedly, closed captions can be translated into many different languages (automatically or by skilled translators), and then can be selected by the viewer based on whatever language works best for them. Open captions cannot.

Open captions cannot be turned off by users who are impacted by them. Closed captions can be set preferentially to whatever the user finds most helpful.

Open captions are also impacted by the font choice, and I’ve noticed a high correlation between “fancy” open captions and really awful typography that isn’t nearly as readable as the user’s selected closed-captioning font, and certainly can’t be changed for folks with dyslexia or other reading impairments.

If a video has both open and closed captions, the open captions will often impact the readability of the closed captions, by having background elements that are jumping around right behind where the closed captions appear!

One reason people often state that they want to use open captions is that they want to add “visual interest” to a visually-boring video. I’d posit that if you have a video that’s of that nature, then adding more visual interest (especially in the form of flashy captions) will only make it harder for people who need captions to understand them. Instead, here’s some things you can do:

• Have multiple camera angles that you switch/fade between (ideally with a rhythm that matches the speech)
• Zoom in on the video and do a slow pan over a few seconds (which can also be super effective when combined with a fade between camera angles)
• Cut to (appropriate) B-roll
• Add an infographic that emphasizes the point of what the speaker is saying

Also, existing footage can be modified for emphasis. One of my favorite tricks for that is a “focus pull” effect, where an area of interest remains focused and the rest of the frame gets blurred. You can apply this to B-roll, to still images, or even to the primary footage (to bring attention to the speaker’s face, for example). This is also easy to overdo, though, so use it sparingly!

(If you aren’t sure what I’m talking about, I used a focus pull effect a lot — probably too much, honestly — in the music video for Wilfa Svart Uniform, to add visual emphasis to some stock/borrowed footage and to “animate” some stock photography. Final Cut Pro calls the effect “focus” and I believe Premiere and DaVinci Resolve have similar effects as well, although I’m not as familiar with their effect libraries.)

Anyway. This isn’t to say that you should never have words appearing on the screen that mirror speech; it can be used to add emphasis at key moments. But please err on the side of user control and not making everything into an attention-grabbing visual nightmare. Our brains are already oversaturated as it is.