A Harmony of Understanding

“We were using music as a way to communicate the connectivity among species in a marine sanctuary,” Terry Wolkowicz explains.

“We all know music has been used to replicate the sounds of animals, right? Their type of communication. But we were able to use music to imitate the specific way in which the animal moves through its environment,” she says.

The deep notes of a cello dip and flow, a violin strains in the higher registers, while a triangle dings in the background, bubbling through the series of notes.

This resonant, orchestral piece is more than an informational track; it is a synchronised multimodal learning experience for the blind, low-vision, or disabled, designed to mimic and display the feeding behaviours of a whale and her calf.

And it’s hauntingly beautiful.

Listen.

Founded during the pandemic, Sound Explorations is a non-profit passion project of pianist and musical educator Terry Wolkowicz and her team. Originally the Education Director for the New Bedford Symphony Orchestra in Massachusetts, USA, lockdown meant discovering new ways of reaching out to their community through sound. Science museums, foundations, and marine conservation groups requested the orchestra’s help in conveying scientific data.

But how did Terry become such an advocate for whale conservation in the first place?

She smiles, “It was the best day of my life.” The day she saw the whales.

During her early work with researchers at the National Oceanic and Atmospheric Association at Stonewall Bank, Terry was invited to assist while they tagged humpback whales. As each animal surfaced, their tiny boat would nestle up to the feeding whales and gently attach a suction cup tag.

“It's one thing to have knowledge. It's another to have something happen in your life and create a strong, emotional, crystallising moment—being so close to a whale was life-changing,” she says, suddenly serious.

“It opened my eyes to the importance of caring about all parts of the environment. So it was not that I learned something about climate change in that moment on the water. Rather, what I understood about climate change all of a sudden became really important.”

Imagine a jellyfish. At first, it appears to drift in the ocean currents, with no agency in its movement. But then, as you watch, the bell of the jellyfish, the dome, expands and then contracts, using the motion of the vortices it creates to power itself through the water.

How do we create a multimodal sensory experience that might allow someone to feel the movement of the jellyfish? Terry and her team exemplify a specific field called data sonification, which involves representing data through sound. While sonification can be as simple as a pitch rising or falling in relation to an ascending or descending data point, it can also be used to demonstrate complex relationships through music.

“If you want to communicate a lot of different things at once, music allows you to do that,” Terry explains. “When multiple musical parts are available, our ears can decide to focus on one of those musical parts or hear all of them in combination to hear how they interact. This has allowed us to represent some very complex ecosystems or environments by combining musical parts within a piece.”

Terry is modest about her grasp of complicated scientific principles, describing how it took her repeat readings (17 times, she claims) to fully understand a scientific paper on jellyfish locomotion (though her ability to break down complex ideas would demonstrate her skills as an educator).

“It was pretty dense, and yet, in a musical representation, in 15, 20 seconds, you could understand what's going on, but in a much more accessible way,” she says. And this facilitation of accessibility and inclusion through music is what Sound Explorations is all about.

In small groups of no more than ten, disabled and low-vision students enter a room at Boston’s Museum of Science. Spread across the tables are several tactile ‘sculptures,’ swirling wires and forms studded with corkboard, bumps, and textured elements, each one resembling a miniature roller coaster.

The students take their seats, and in walks a professional musician who will sit with one child at a time. As the students run their hands over the sculpture, the musician accompanies their movements, immersing the listener in a responsive musical sonification of a whale’s feeding pattern.

“We’ve done this with clarinet and bass clarinet. The bass clarinet's awesome because it has a low resonance, which kind of feels like a whale. Also, string instruments, we've used cello, viola, and violin. They work really well, especially because some people go very slowly across the model, and the musicians don't need to take a breath,” Terry laughs.

Current sculptures show the rolling feeding behaviour of a humpback whale, the double loop bubble net feeding behaviour of a humpback whale, and then two foraging behaviours of the North Atlantic Right Whale.

These talented musicians come from the New Bedford Symphony Orchestra, the New England Conservatory of Music, and other local orchestras. However, they aren’t presented with a set of printed sheet music; instead, they must improvise based on the user’s movements.

“We obviously give them a little education about how we interpret certain behaviours and how we sonify it, but in the end, it's the musician who is creating something with that person. Sometimes kids go forward, and then their whale starts to go backwards,” she smiles. “You got to go with them.”

“Sometimes kids go really fast, and you have to play fast. Sometimes adults will go really slowly because they want to feel every bump and curve,” she laughs again.

Emily Grossman is the artistic director who designs and constructs each sculpture, capturing the dives and breaches of various whale species, creating a ‘tactile vocabulary’ that turns touch into sound and sound into comprehension.

“We can use many other musical elements to show not only changes over time but also relationships, patterns, and sequences of events, even the idea of biodiversity or changes in an ecosystem,” Terry says. “By using not just pitch but rhythm, texture, timbre, and harmony, we can really depict the full representation of the data more precisely.”

And as for making an impact, Terry knew that would happen by bridging scientific understanding with action.

“It's the difference between accessibility and inclusion. Accessibility means the experience itself could be accessible to differently-abled people. But the idea of inclusion means you value their participation in decision-making and problem-solving and being an advocate.”

“To me, this is crucial because when I've met with blind and low-vision children and adults to really understand how these whales are behaving, moving, and foraging for food, it brings them into the conversation,” she continues.

“They feel more knowledgeable and engaged to now participate as a stakeholder in protecting and conserving our environment.”

However, the takeaways from the Sound Explorations community work are what Terry focuses on the most. “There's this one little boy, Romeo, who said he was so happy because it showed him he wasn't alone,” she says, speaking about their event at the Boston Science Museum. “And that day, he wanted to tell me how he made a new friend.”