After Sound Is Put in Brackets

When language enters subtitles, it usually feels as if it is being recorded. When sound enters subtitles, it often feels as if it is passing through a checkpoint. Human speech can be arranged sentence by sentence into words; the sounds in a room must first explain why they deserve to appear, why they would interrupt a sentence, or cover it over completely. The problem is not that [music] is too short, but that this pair of brackets is like a narrow gate, deciding which sounds can be handed over to the eyes.

So when [music] pops up in the corner of the screen, I pause for a moment. The person who can hear may be surrounded by a very specific sound: a song leaking out of an old radio, low frequencies pressing closer from beneath the floor, an ill-timed love song playing through a restaurant speaker. The subtitle offers only a word that has been wiped clean, as if someone has wrung the whole room into a small label and stuck it to the edge of the image.

To call this simply a loss would be too easy. For Deaf and hard-of-hearing viewers, subtitles are not supplementary notes, nor are they a damaged photocopy of the auditory original; they are already a way of entering the same event. Without [music], that stretch of sound would not even be handed over to the eyes. The issue is not that subtitles are insufficiently like sound, but that once sound becomes text, it is often forced to withdraw from atmosphere and become a form of useful information.

Those who can hear do not have to turn every sound into information. The refrigerator hums low by the wall, the bottom of a cup touches the tabletop, someone in the hallway drops keys into a bowl; these sounds can have no use, and simply allow a place to continue resembling itself. But once sound enters subtitles, it has usually already passed through a filter. APPLAUSE DROWNS SPEECH: applause drowns out speaking. GUNFIRE: no need to explain who is shooting at whom. Those labels are so calm they are almost brutal, but they are also honest: sometimes subtitles cannot return a sentence to you, and can only tell you how language was defeated by sound.

Automatic captioning exposes this more directly. It is too accustomed to first looking for what people are saying; as for the room, the wind, the distant instrument, they are often noticed a step too late. But human captioning is not working on a sheet of paper where one can write slowly, either. It faces a few seconds, two lines of text, and a pair of moving eyes. Roughness sometimes comes from laziness, and sometimes from the compression the medium has to perform.

Conversely, sound should not simply be pushed wholesale into the center of the screen. If subtitles love explanation too much, they preempt the viewer’s feeling: writing a passage of melody as [gentle romantic music plays] may be more eager than [quiet violin melody], but it is also more arbitrary. For someone who enters a film only through subtitles, too much explanation of sound can become another kind of noise. Better subtitles are not necessarily more literary; they leave the viewer a little less uniformly arranged.

The most moving moments, instead, appear when the brackets have not given up on texture. [heart throbbing] thud-dub, thud-dub. It does not explain the heartbeat as nervousness, but moves a small piece of rhythm before the eyes. [wet towel slapping] thrack is not an action description either, but the wet, heavy smack of a blow. Here, writing does not pretend to restore sound; it simply remakes a readable beat inside the eyes.

Later, when I see [music] again, I no longer find it merely impoverished. It is like a small slip of paper pasted over a crack in the door, telling the person reading it: there really is sound in the room. It is just that the door has not opened. The slip of paper trembles slightly. That sound may be very near, or it may be beyond rain and walls, dragging a bit of temper as it passes somewhere outside the screen. The eyes rest on the brackets, wait a moment, and then keep reading.