George Tzanetakis

George Tzanetakis

Rise of the robot musicians

UVic professor George Tzanetakis on track to create computer music systems that can jam with humans

The thing about robot musicians is they’ll never trash a hotel room, show up to a gig drunk or break up a band after meeting an eccentric performance artist.

At least not yet.

The music-computer lab of George Tzanetakis at the University of Victoria is ground zero for the rise of robot musicians. But don’t think of it like anthropomorphic C-3PO strumming a mandolin – it’s more like a series of mallets linked to solenoids on a drum, backed by sophisticated software. Tzanetakis effectively wants to teach music to a computer, and for the computer to pick up on musical cues while jamming with humans.

“When you play sound, a musician hears what is happening. We are trying to add the ability to understand music to an artificial agent that performs,” says Tzanetakis, an assistant professor in the department of computer science, and in the department electrical and computer engineering – and the school of music.

“The idea is to make the system musically intelligent, to have robotic musicianship.”

It remains early days for robot jazz bands. Mechanical musicians are largely limited to percussion instruments due to increasing engineering complexity as each mallet and its actuator is added to the mix, and to the software that crunches the music in real time to adjust the tempo and volume.

“We want the musicality of the system to learn how to slow down, speed up, understand if it is too loud. We are still working on making sure it doesn’t get drunk,” Tzanetakis jokes.

Of course computers, even advanced learning systems, don’t understand music or sound or much of anything else – they just crunch data in sophsticated ways. Gabrielle Odowichuk, who worked in UVic computer-music lab as a masters student, said completely computer-driven music seems unlikely. Computers still can’t connect on an emotional level.

“You need a human component or its boring. All robots need a human component,” says Odowichuk who worked on gestural control of sound. “Often you can program in more randomness to make it more human. But when it comes to conveying something and connecting with an audience, computers lack expressiveness.”

Tzanetakis sees robots and computer musicality as the natural evolution in how technology influences how music is made. To him, it’s no different than the person who first strung wires and called it a piano, or plugged in a wah-wah pedal.

“Using computers and robots is the same process,” he says. “Musicians accept synthesizers, drum machines and DJ sets. If anything a robot is more innocent than a drum machine. It’s just another technical dimension.”

How this work will eventually influence the broader music world is hard to predict. Tzanetakis, a 37-year-old native of Greece who earned his PhD in computer science from Princeton University in 2002, tends to be about five years ahead of current popular technology.

Tzanetakis’ early work on acoustic signal processing laid the groundwork for popular apps such as Shazam, which can recognize songs by holding a smartphone up to a speaker. He’s helped develop melody-matching systems that identify songs by users humming or singing. His music sensing algorithms are at the heart Smule’s AutoRap app, which creates a rap song out of any set of sounds, and the Ocarina app, which transforms an iPhone into a flute.

Of course, when he began working on what’s called audio fingerprinting – systems that analyze, dissect and find patterns in acoustic signals – smartphone technology didn’t exist.

“If I do my job right, stuff I’m doing now will be commercial in about five years,” he says. “What is commercial now is stuff myself and others were doing 10 years ago. Ten years ago there was no iPod, no iTunes. The world was a different place.”

Tzanetakis expects the next big creative leaps to involve computer systems that can analyze and extract individual instruments and voices from complex music scores. The human ear can pick our individual conversations in a noisy party – computers cannot.

“It’s really a hard problem, far from being solved, but is actively researched,” he said.

Despite being at the forefront of intertwining computers and music, Tzanetakis, a jazz musician and pianist before he became a computer scientist and engineer, admits he doesn’t always practice what he preaches.

“I’m a sax player. I have typically avoided playing with technology.”

editor@saanichnews.com

 

 

Just Posted

Greater father involvement in the home leads to improved childhood development and increased marital satisfaction, says expert. (Black Press Media file photo)
University of Victoria researcher finds lack of father involvement a drag on gender equality

Working women still taking on most child and household duties in Canada

New COVID-19 cases on Vancouver Island by local health area for the week of June 6-12. (B.C. Centre for Disease Control image)
New COVID-19 cases up on Island, but health officials say trends going right way

There were 22 new COVID-19 cases in Greater Victoria last week after just four the week before

Emergency health services treated a person after they were blocking traffic at the intersection of Fort and Douglas Streets on June 17. (Evert Lindquist/ News Staff)
Victoria intersection traffic returns to normal after protester blocked roadway

A person in a motorized wheelchair was blocking the intersection at Fort and Douglas Streets

Eric White’s roadside farm stand in Metchosin sits stocked with produce. (Photo courtesy of Eric White)
Fledgling Metchosin farmer frustrated by thefts from stand

Eric White said every dollar made at the roadside helps sustain his farm

Saanich police took a suspect into custody after a store employee on Cedar Hill Cross Road was assaulted Wednesday afternoon. (Black Press Media file photo)
Employee assaulted at Saanich store after asking suspected shoplifters to leave

June 16 incident saw worker taken to hospital with non-life-threatening injuries

People line up to get their COVID-19 vaccine at a vaccination centre, Thursday, June 10, 2021 in Montreal. THE CANADIAN PRESS/Ryan Remiorz
Vaccines, low COVID case counts increase Father’s Day hope, but risk is still there

Expert says people will have to do their own risk calculus before popping in on Papa

(Black Press Media file photo)
POLL: When was the last time you visited the mainland?

The films are again lighting the screens at local theatres, the wine… Continue reading

Anyone with information on any of these individuals is asked to call 1-800-222-TIPS (8477) or visit the website victoriacrimestoppers.ca for more information.
Greater Victoria Crime Stoppers wanted list for the week of June 15

Greater Victoria Crime Stoppers is seeking the public’s help in locating the… Continue reading

The Co-op gas station at Whiskey Creek is burning after a camper van exploded while refueling just before 4 p.m. on Thursday, June 17, 2021. (FACEBOOK PHOTO)
Exploding camper van torches Highway 4 gas station between Qualicum Beach and Port Alberni

Highway traffic blocked after Whiskey Creek gas station erupts into flames

Helen Austin performing with Trent Freeman at the 2018 Vancouver Island MusicFest. Austin is one of the many performers listed for the 2021 event.
Vancouver Island MusicFest goes virtual for 2021

Black Press to stream 25 hours of programming July 9-11

FILE – A science class at L.A. Matheson Secondary in Surrey, B.C. on March 12, 2021. (Lauren Collins/Surrey Now Leader)
Teachers’ union wants more COVID transmission data as B.C. prepares for back-to-school

BCTF says that details will be important as province works on plan for September

Provincial health officer Dr. Bonnie Henry outlines B.C.’s COVID-19 restart plan, May 25, 2021, including larger gatherings and a possible easing of mandatory masks on July 1. (B.C. government photo)
B.C. records 120 new COVID-19 cases, second vaccines accelerating

Lower Pfizer deliveries for early July, Moderna shipments up

A Heffley Creek peacock caught not one - but two - lifts on a logging truck this month. (Photo submitted)
Heffley Creek-area peacock hops logging trucks in search of love

Peacock hitched two lifts in the past month

The Calgary skyline is seen on Friday, Sept. 15, 2017. THE CANADIAN PRESS/Jeff McIntosh
2 deaths from COVID-19 Delta variant in Alberta, 1 patient was fully immunized

Kerry Williamson with Alberta Health Services says the patients likely acquired the virus in the hospital

Most Read