Record companies are tracking download and search data to predict which new songs will be hits. This has been good for business—but is it bad for music?
We noticed that you have an
In 2000, a Stanford Ph.D. named Avery Wang co-founded, with a couple of business-school graduates, a tech start-up called Shazam. Their idea was to develop a service that could identify any song within a few seconds, using only a cellphone, even in a crowded bar or coffee shop.
At first, Wang, who had studied audio analysis and was responsible for building the software, feared it might be an impossible task. No technology existed that could distinguish music from background noise, and cataloging songs note for note would require authorization from the labels. But then he made a breakthrough: rather than trying to capture whole songs, he built an algorithm that would create a unique acoustic fingerprint for each track. The trick, he discovered, was to turn a song into a piece of data.
Shazam became available in 2002. (In the days before smartphones, users would dial a number, play the song through their phones, and then wait for Shazam to send a text with the title and artist.) Since then, it has been downloaded more than 500 million times and used to identify some 30 million songs, making it one of the most popular apps in the world. It has also helped set off a revolution in the recording industry. While most users think of Shazam as a handy tool for identifying unfamiliar songs, it offers music executives something far more valuable: an early-detection system for hits.
By studying 20 million searches every day, Shazam can identify which songs are catching on, and where, before just about anybody else. “Sometimes we can see when a song is going to break out months before most people have even heard of it,” Jason Titus, Shazam’s former chief technologist, told me. (Titus is now a senior director at Google.) Last year, Shazam released an interactive map overlaid with its search data, allowing users to zoom in on cities around the world and look up the most Shazam’d songs in São Paulo, Mumbai, or New York. The map amounts to a real-time seismograph of the world’s most popular new music, helping scouts discover unsigned artists just as they’re starting to set off tremors. (The company has a team of people who update its vast music library with the newest recorded music—including self-produced songs—from all over the world, and artists can submit their work to Shazam.)
“We know where a song’s popularity starts, and we can watch it spread,” Titus told me. Take, for example, Lorde, the out-of-nowhere sensation of 2013. Shazam’s engineers can rewind time to trace the international contagion of her first single, “Royals,” watching the pings of Shazam searches spread from New Zealand, her home country, to Nashville (a major music hub, even for noncountry songs), to the American coasts, pinpointing the exact day it peaked in each of nearly 3,000 U.S. cities.
Shazam has become a favorite app of music agents around the country, and in February, the company announced that it would get into the music-making business itself, launching a new imprint under Warner Music Group for artists discovered through the app.
Shazam searches are just one of several new types of data guiding the pop-music business. Concert promoters study Spotify listens to route tours through towns with the most fans, and some artists look for patterns in Pandora streaming to figure out which songs to play at each stop on a tour. In fact, all of our searching, streaming, downloading, and sharing is being used to answer the question the music industry has been asking for a century: What do people want to hear next?
It’s a question that label executives once answered largely by trusting their gut. But data about our preferences have shifted the balance of power, replacing experts’ instincts with the wisdom of the crowd. As a result, labels have gotten much better at understanding what we want to listen to. This is the one silver lining the music industry has found in the digital revolution, which has steadily cut into profits. So it’s clearly good for business—but whether it’s good for music is a lot less certain.
Earlier this year, Patch Culbertson, a scout for Republic Records, sat in his New York office and opened the Shazam map on his iPhone. Republic Records is the most data-driven major label in the music business (even an executive at a rival label described Republic as the gold standard for using analytics in scouting and marketing), and Culbertson in particular has proved to be a star at the company.
Culbertson wanted to check up on SoMo, an R&B singer from Denison, Texas, whom Culbertson had helped sign last year. Culbertson zoomed in on Victoria, Texas, a small city between Corpus Christi and Houston, where one of the radio stations had started playing a SoMo single called “Ride.” Although a town of just 63,000 won’t launch a national hit by itself, Culbertson was using Victoria as a sort of testing ground to determine whether the song would resonate with listeners. “ ‘Ride,’ ” he told me, “is the No. 1 tagged song in Victoria.”
Pop music is a sentimental business, and predicting the next big thing has often meant being inside that crowded bar, watching a young band connect with the besotted, swaying throng. But now that new artists are more likely to make a name for themselves on Twitter than in a Nashville club, Culbertson is finding that the chair in front of his computer might be the best seat in the house.
New tools may soon further diminish the importance of actually hearing artists perform. Next Big Sound, a five-year-old music-analytics company based in New York, scours the Web for Spotify listens, Instagram mentions, and other traces of digital fandom to forecast breakouts. It funnels half a million new acts through an algorithm to create a list of 100 stars likely to break out within the next year. “If you signed our top 100 artists, 20 of them would make the Billboard 200,” Victor Hu, a data scientist with Next Big Sound, told me. A 20 percent success rate might sound low, until you gaze out at the vast universe of new music and try to pick the next Beyoncé.
Last year, the company unveiled a customizable search tool called Find, which, for a six-figure annual subscription, helps scouts mine social media to spot artists who show signs of nascent stardom. If, for example, you wanted to search for obscure bands with the fastest-growing followings on Twitter, Find could produce a list within seconds.
The top 1 percent of bands and solo artists now earn 77 percent of all revenue from recorded music.
The company has discovered that some metrics, such as Facebook likes, are unreliable indicators of a band’s trajectory, while others have uncanny forecasting power. “Radio exposure, unsurprisingly, is the most important thing,” Hu says. It remains the best way to introduce listeners to a new song; once they’ve heard it a few times on the radio, they tend to like it more. “But we discovered that hits to a band’s Wikipedia page are the second-best predictor.” Wikipedia searches are revealing for the same reason Shazam searches are. While getting a song on the radio ensures that people have heard it, Culbertson says, “Shazam tells you that people wanted to know more.”
To get a song on the radio in the first place, music labels confront a paradox: How do you prove that it will be a hit before anyone has heard it? DJs consider unfamiliar songs “tune-outs,” because audiences tend to spurn new music. In the past, labels sometimes pressured or outright bribed stations to promote their music. Songs became hits because executives decided they should be hits.
But radio, too, has come to rely more on data, and now when label executives pitch a station, they’re likely to come armed with spreadsheets. The search for evidence of a song’s potential has become exhaustive: you can’t just track radio data, or sales, or YouTube hits, or Facebook interactions, or even proprietary surveys and focus groups. To persuade a major radio station to play a new song, labels have to connect all these dots.
“The idea that DJs are just picking songs because they like them is so antiquated,” says Radha Subramanyam, the executive vice president of insights, research, and analytics at iHeartMedia (formerly Clear Channel), the nation’s largest owner of FM stations. iHeartMedia consults companies like Shazam to figure out which songs are going viral. Nielsen Audio, another data firm that has partnered with the company, offers thousands of listeners cash or gift cards to wear devices called Portable People Meters that track which radio stations people are tuning in to. To know when listeners are growing tired of a song, iHeartMedia conducts weekly surveys using a database of 1.5 million people.
Perhaps iHeartMedia’s most interesting partner in the search for pop music’s next big thing is a 12-year-old subsidiary called HitPredictor, which, true to its name, predicted 48 of the top 50 radio hits last year. Before a song debuts on a major chart—Top 40, urban, country, or alternative—HitPredictor plays key sections for its online database of listeners and rates their responses. Any song that scores above a 65 is considered a possible breakout, though above that threshold, the highest-scoring songs don’t always do best. (Meghan Trainor’s debut single, “All About That Bass,” scraped by with a 68.97 rating but went on to become the top song in the country this fall.)
All of this number crunching is aimed at keeping listeners’ fingers off the dial. “It’s not about eliminating the human element from radio, but rather presenting the most human element—the reaction of audiences—more clearly than
ever,” Jay Frank, the owner and CEO of DigSin, a digital record label (it sells music strictly through downloads—no CDs), told me. “This might be the most populist moment in radio history.”
A similar revolution has occurred in the music charts. Take the Billboard Hot 100, which has counted down the top songs in America since 1958. For decades, Billboard had to rely on record-store owners and radio stations to report the most-bought and most-played songs. Both parties lied, often because labels nudged or bribed them to plug certain records, or because store owners didn’t want to promote albums they no longer had in stock. The entire industry was biased toward churn: labels and stores wanted songs to enter and exit the charts quickly so they could keep selling new hits.
The Hot 100 matters because it doesn’t just reflect listener preferences, it also shapes them. In a groundbreaking 2006 study on the influence of song rankings, three researchers at Columbia University showed that popularity can be a self-fulfilling prophecy. The researchers sent participants to different music Web sites where they could listen to dozens of tracks and download their favorites. Some sites displayed a ranking of the most-downloaded songs; others did not. Participants who saw rankings were more likely to listen to the most-popular tracks.
The researchers then wondered what would happen if they manipulated the rankings. In a follow-up experiment, some sites displayed the true download counts and others showed inverted rankings, where the least-popular song was listed in the No. 1 spot. The inverted rankings changed everything: previously ignored songs soared in popularity, and previously popular songs were ignored. Simply believing, even wrongly, that a song was popular made participants more likely to download it.
Billboard replaced its honor system with hard numbers in 1991, basing its charts on point-of-sale data from cash registers. “This was revolutionary,” says Silvio Pietroluongo, Billboard’s current director of charts. “We were finally able to see which records were actually selling.” Around the same time, Billboard switched to monitoring radio airplay through Nielsen.
When that happened, hip-hop and country surged in the rankings and old-fashioned rock slowly began to fade—suggesting that perhaps an industry dominated by white guys on the coasts hadn’t paid enough attention to the music interests of urban minorities and southern whites.
Another sea change came in the mid-2000s, when Billboard started tracking music streaming and downloads. Songs that weren’t label-picked singles, like the Black Eyed Peas’ “My Humps” in 2005, began outperforming the tracks that executives expected to do well. “Deep cuts”—songs that labels didn’t hype but that fans nonetheless loved—used to fly under the radar. (There is no evidence that Led Zeppelin’s “Stairway to Heaven,” one of the most famous rock songs of all time, was ever played on the radio in the years immediately after its release, and it never cracked the Hot 100.) But because the industry can now track what people are listening to, any song that catches on can become a hit.
Everyone I spoke with about the Hot 100—label and radio executives, industry analysts, and other journalists—agreed with Jay Frank’s assessment that consumers have more say than they did decades ago, when their tastes were shaped by the hit makers at labels. But here’s the catch: if you give people too much say, they will ask for the same familiar sounds on an endless loop, entrenching music that is repetitive, derivative, and relentlessly played out.
Now that the Billboard rankings are a more accurate reflection of what people buy and play, songs stay on the charts much longer. The 10 songs that have spent the most time on the Hot 100 were all released after 1991, when Billboard started using point-of-sale data—and seven were released after the Hot 100 began including digital sales, in 2005. “It turns out that we just want to listen to the same songs over and over again,” Pietroluongo told me.
Because the most-popular songs now stay on the charts for months, the relative value of a hit has exploded. The top 1 percent of bands and solo artists now earn 77 percent of all revenue from recorded music, media researchers report. And even though the amount of digital music sold has surged, the 10 best-selling tracks command 82 percent more of the market than they did a decade ago. The advent of do-it-yourself artists in the digital age may have grown music’s long tail, but its fat head keeps getting fatter.
Radio stations, meanwhile, are pushing the boundaries of repetitiveness to new levels. According to a subsidiary of iHeartMedia, Top 40 stations last year played the 10 biggest songs almost twice as much as they did a decade ago. Robin Thicke’s “Blurred Lines,” the most played song of 2013, aired 70 percent more than the most played song from 2003, “When I’m Gone,” by 3 Doors Down. Even the fifth-most-played song of 2013, “Ho Hey,” by the Lumineers, was on the radio 30 percent more than any song from 10 years prior.
The reliance on data may be leading to a “clustering” of styles and a dispiriting sameness in pop music.
And not only are we hearing the same hits with greater frequency, but the hits themselves sound increasingly alike. As labels have gotten more adept at recognizing what’s selling, they’ve been quicker than ever to invest in copycats. People I spoke with in the music industry told me they worried that the reliance on data was leading to a “clustering” of styles and genres, promoting a dispiriting sameness in pop music.
In 2012, the Spanish National Research Council released a report that delighted music cranks around the world. Pop, it seemed, was growing increasingly bland, loud, and predictable, recycling the same few chord progressions over and over. The study, which looked at 464,411 popular recordings around the world between 1955 and 2010, found that the most-played music of the new millennium demonstrates “less variety in pitch transitions” than that of any preceding decade. The researchers concluded that old songs could be made to sound “novel and fashionable” just by freshening up the instrumentation and increasing “the average loudness.”
The problem is not our pop stars. Our brains are wired to prefer melodies we already know. (David Huron, a musicologist at Ohio State University, estimates that at least 90 percent of the time we spend listening to music, we seek out songs we’ve heard before.) That’s because familiar songs are easier to process, and the less effort needed to think through something—whether a song, a painting, or an idea—the more we tend to like it. In psychology, this idea is known as fluency: when a piece of information is consumed fluently, it neatly slides into our patterns of expectation, filling us with satisfaction and confidence.
“Things that are familiar are comforting, particularly when you are feeling anxious,” Norbert Schwarz, a psychology professor at the University of Southern California, who studies fluency, told me. “When you’re in a bad mood, you want to see your old friends. You want to eat comfort food. I think this maps onto a lot of media consumption. When you’re stressed out, you don’t want to put on a new movie or a challenging piece of music. You want the old and familiar.”
It would be too simplistic to say that music is racing in a single direction—toward dumber, louder, and more-repetitive pop. Now that labels recognize how popular hip-hop and country really are, they have created innovative new sounds by blending those genres with traditional pop. One of the popular songs of this past summer, “Problem,” combined a dizzy sax hook, ’90s-pop vocals, a whispered chorus, and a female rap verse. It was utterly strange and, for a while, ubiquitous. Greta Hsu, an associate professor at the University of California at Davis, who has done research on genre-blending in Hollywood, told me that although mixing categories is risky, hybrids can become standout successes, because they appeal to multiple audiences as being somehow both fresh and familiar.
Music fans can also find comfort in the fact that data have not taken over the songwriting process. Producers and artists pay close attention to trends, but they’re not swimming in spreadsheets quite like the suits at the labels are. Perhaps one reason machines haven’t yet invaded the recording room is that listeners prefer rhythms that are subtly flawed. A 2011 Harvard study found that music performed by robotic drummers and other machines often strikes our ears as being too precise. “There is something perfectly imperfect about how humans play rhythms,” says Holger Hennig, the Harvard physics researcher who led the study. Hennig discovered that when experienced musicians play together, they not only make mistakes, they also build off these small variations to keep a live song from sounding pat.
The Internet can connect us to an astonishing amount of music—some of it derivative, but much of it wildly experimental, even brilliant. Streaming services like Spotify and Pandora let us sample from music libraries that, decades ago, wouldn’t have fit inside the largest record store in the world. These services aren’t just vast; they’re also searchable and exquisitely personal. “One thing about Pandora that isn’t obvious to people who use our service is that it isn’t just one algorithm,” Eric Bieschke, the company’s chief scientist, told me. “We have dozens and dozens of algorithms that connect people to music in different ways, like genre, and popularity, and repetitiveness. Then we have a meta-algorithm that directs all of the algorithms, like a conductor standing in front of a symphony that’s only playing for one person.”
But while fans can burrow deep into rabbit holes of esoterica, “Today’s Top Hits” is still the No. 1 playlist on Spotify, and Pandora’s most popular station is “Today’s Hits.” Even when offered a universe of music, most of us prefer to listen to what we think everyone else is hearing.