PDF of RailsConf Spotify Keynote

Transcription

Music + Data = Fun!
Some experiments in how we can use music data
to help improve the listening experience
RailsConf
May 6, 2016
Paul Lamere
The Aerosmith Anomaly
Relative listens per day
Relative Number of Listens
“Don’t want to miss a thing” - Aerosmith
Relative streams per day
Strange listen spikes detected
“Close Shave” with Asteroid 2012 DA1
‘Potentially Hazardous Asteroid’ EM26 near miss
Rosetta spacecraft orbits Comet 67P
Philae landing site selected on Comet 67P
Robotic lander Philae touched down
on Comet 67P
Robotic lander Philae touched down
on Comet 67P
https://insights.spotify.com/us/2014/11/21/aerosmith-rosetta-comet/
A little bit about the speaker
Why do we care?
I’ve got 30 million songs in my pocket.
Now what?
Data about music
Music Metadata artists, albums songs, labels Cultural Data What people are saying about music Listener Data Who listened to what and when Acoustic Data What the music sounds like Data about music
Music Metadata artists, albums songs, labels Cultural Data What people are saying about music Listener Data Who listened to what and when Acoustic Data What the music sounds like Music Metadata artists, albums songs, labels •
5 Million artists •
10 Million albums •
40 Million tracks name, biographies, images, years active, artist location,
band members, genre
title, year of release, artist, genre, track list, related
releases. album art, producer, label
title, year of release, duration, genre,
The most boring data
Music Metadata
Many challenges with music metadata
A simple query:
What are the songs on the album “White Christmas” by Bing Crosby
Let’s pause for a quiz
Why is this formula troublesome for music recommendation and discovery?
(ΔMī¹=αΣDi[n][ΣFij[n-1]+Fexti[[n̄¹]])
Let’s pause for a quiz
Why is this formula troublesome for music recommendation and discovery?
Because it is the name of a
song by Aphex Twin
(ΔMī¹=αΣDi[n][ΣFij[n-1]+Fexti[[n̄¹]])
OMG Metadata
OMG Metadata
The The
OMG Metadata
The The
Duran Duran Duran
OMG Metadata
The The
Duran Duran Duran
DJ Donna Summer
OMG Metadata
The The
Duran Duran Duran
DJ Donna Summer
GL SS †33†H
OMG Metadata
The The
Duran Duran Duran
DJ Donna Summer
GL SS †33†H
!!!
OMG Metadata
The The
Duran Duran Duran
DJ Donna Summer
GL SS †33†H
!!!
†††
OMG Metadata
The The
Duran Duran Duran
DJ Donna Summer
GL SS †33†H
!!!
†††
///▲▲▲\\\
OMG Metadata
The The
Duran Duran Duran
DJ Donna Summer
GL SS †33†H
!!!
†††
///▲▲▲\\\
▼□■□■□■
OMG Metadata
The The
Duran Duran Duran
DJ Donna Summer
GL SS †33†H
!!!
†††
///▲▲▲\\\
▼□■□■□■
Various Artists
OMG Metadata
The 22 bands named Eclipse
Experiment #1
Answer a very important question on Quora
Have band names been getting longer?
let’s use the some data to find out
• Collect the top 500 artists for each 5 year window
(via Echo Nest or Musicbrainz)
• Calculate the average name length for period.
Have artist names been getting longer?
Let’s find out with data!
The code is shorter than Zach’s answer
Year
NO
1955 - 1959!
Average name length
The average length of artist names peaked in the period from 1955-1959.
Average name length was 2 characters longer in 1955 than in 2012
Band names from 1955-1959
44 Van McCoy & The Soul City Symphony Orchestra
35 Academy of St. Martin in the Fields
33 The Clancy Brothers & Tommy Makem
33 Cliff Bennett & The Rebel Rousers
32 Maurice Williams and The Zodiacs
31 Herb Alpert & The Tijuana Brass
30 Smokey Robinson & The Miracles
30 Little Anthony & The Imperials
29 Van Morrison & The Chieftains
29 Nelson Riddle & His Orchestra
29 George Clinton and Parliament
29 Frankie Lymon & The Teenagers
29 Clancy Brothers & Tommy Makem
Longest band names
51
48
47
46
44
43
43
41
39
37
37
37
36
36
2010
1994
1967
1993
1959
2000
1950
1981
1972
2010
2005
1999
2009
1977
Tim and Sam’s Tim and the Sam Band with Tim and Sam
...And You Will Know Us By The Trail Of The Dead
Charles Wright & The Watts 103rd St Rhythm Band
The Presidents of the United States of America
Van McCoy & The Soul City Symphony Orchestra
Richard Cheese & Lounge Against The Machine
Benedictine Monks of Santo Domingo de Silos
Emir Kusturica & The No Smoking Orchestra
Afrika Bambaataa & The Soul Sonic Force
Antoine Dodson & The Gregory Brothers
Edward Sharpe and the Magnetic Zeroes
Someone Still Loves You Boris Yeltsin
Sten Anderson and the Hungry Fathers
Gloria Estefan & Miami Sound Machine
Data about music
Music Metadata artists, albums songs, labels Cultural Data What people are saying about music Listener Data Who listened to what and when Acoustic Data What the music sounds like Cultural Data What people are saying about music Artist Bios
The web
Playlists
Reviews
Blogs
Lyrics
Forums
Events
Social sites
Understanding an artist
black metal
powerful
metal
extreme
melodic black metal
gothic
run-of-the-mill atmospheric
strong
unique
symphonic epic
orchestration
Norwegian fast
melodic
heavy
famous
Cultural artist similarity
Artist Bios
The web
Reviews
Blogs
Lyrics
Forums
Events
Social sites
Playlists
Cultural artist similarity
pop dance diva sexy female
guilty pleasure teen 00s rock
california dance-pop hot 10s
90s Norway satanic 10s fast
dark 00s heavy melodic
symphonic metal epic
extreme gothic black-metal
Experiment #2
Exposing a listener to new music
Experiment #2
Exposing a listener to new music
The extreme edition
Help a Katy Perry fan
to Dimmu Borgir
l
Help a Katy Perry fan listen to Dimmu Borgir
female metal goth 10s emo
diva dark gothic rock
numetal guilty pleasure
melodic pop epic 00s
Katy Perry
Dimmu Borgir
http://www.stanford.edu/~dgleich/demos/worldofmusic/WorldOfMusic.html
Katy Perry
Evanescence
Dimmu Borgir
Katy Perry
Evanescence
Dimmu Borgir
Data about music
Music Metadata artists, albums songs, labels Cultural Data What people are saying about music Listener Data Who listened to what and when Acoustic Data What the music sounds like Listener Data
Listener Data Who listened to what and when • User / Song/ action / timestamps
•
•
Action: play, skip, favorite, rate, ban, volume change, etc.
10 million listeners generate one
billion data points per week Experiment #3
Who are the most passionate fans?
Dubstep Fan
Metal heads
Passion Index - Average plays per fan
Two artists - each with 1,000,000 plays
200, 000 fans
5 plays per fan
Low Passion
10,000 fans
100 plays per fan
High Passion
Data is this example is fictitious and does not represent the real fans or plays for either of these two fine artists
Passion Index
Passion Index
High plays per listener
Passion Index
Low plays per listener
Who has the most passionate fans?
Average 115 plays per fan
Average 115 plays per fan
Passion Index
passionate
Metal heads are
passionate
music fans
9 out of the top 20 are metal bands. Zero are dubstep
Passion Index
passionate
Metal heads are
passionate
music fans
9 out of the top 20 are metal bands. Zero are dubstep
Passion Index
Who has the least
passionate fans?
Passion Index
Who has the least
passionate fans?
Passion Index
Who has the least
passionate fans?
Passion Index
Who has the least
passionate fans?
Average only 5 plays per ‘fan’
Passion Index
Who has the least
passionate fans?
Passion Index
Who has the least
passionate fans?
Experiment #4
What can we learn from two
billion playlists?
Top playlist names on Spotify
Rap
Country
House
Rock
Music
Chill
Party
Workout
Gym
Roadtrip
Rap
Country
House
Rock
Music
Chill
Party
Workout
Gym
Roadtrip
17 of the top 100 playlist names are genre related
Rap
Country
House
Rock
Music
Chill
Party
Workout
Gym
Roadtrip
41 of the top 100 playlist names are context related
Rap
Country
House
Rock
Music
Chill
Party
Workout
Gym
Roadtrip
41 of the top 100 playlist names are context related
Context is the new genre
Top context-related playlists
Chill Party Workout Relax Gym Dance Sleep Pregame Car
Feel Good Roadtrip Summer Work Study Sex Pump up Night
Drive Training Gaming Shower Ski Snowboarding Cafe
Homework Twerk On the road Motivation School 420 Dinner
Running Friends Sad XC School Yoga Winter Coffee Sleepy Flow
Girls Night Commute Drinking Crossfit Break Up Inspiration
Smoking Romance Afterski Preparty Beach Cardio Hike Fly
Afternoon Happiness Training Walk Predrinks Office Cry
Wake up Travel Fitness Focus Cleaning Cruising Lifting Evening
Rainy Day Autumn Train Love Worship Depression Morning
Training and workout
Mood
Travel
Romance
Time
Focus
Socializing
Context is important, but …
Context is hard!
Context is hard!
Let’s find music for a particular context by mining tracks
from the 2 billion existing playlists
Context is hard!
Let’s find music for a particular context by mining tracks
from the 2 billion existing playlists
“Create a playlist of mainstream tracks good for running for an 55 year-old male”
The Playlist Miner
Query: Distinctive mainstream running songs for 55+ year-old male
running
morning run
running tunes
2 Billion
Playlists
songs for my run
running
fast run
tuesday run
“running”
10,000 running playlists
The Playlist Miner
running
morning run
100,000 ranked running songs
1.
2.
3.
4.
5.
running tunes
songs for my run
running
fast run
ay run
running playlists
aggregate tracks
Filter tracks by
demographics,
popularity, etc.
R
B
G
E
I
The Playlist Miner
1.
2.
3.
4.
5.
Filter tracks by
demographics,
popularity, etc.
Running on Empty - Jackson Browne
Born to Run - Bruce Springsteen
Gonna Fly Now (Rocky) - Bill Conti
Eye of the Tiger - Survivor
I Ran (So Far Away) - Flock of Seagulls
Top 55+ year-old male Running tracks
1.
2.
3.
4.
5.
1.
2.
3.
4.
5.
1.
2.
3.
4.
5.
Top 18-24 year-old female Running tracks
1.
2.
3.
4.
5.
Runaway Baby - Bruno Mars
I Just Wanna Run - The Downtown Fiction
Stronger - Kelly Clarkson
Fighter - Christina Aguilera
Every chance we get we run - David Guetta
1.
2.
3.
4.
5.
1.
2.
3.
4.
5.
Top 55+ year-old male Roadtrip tracks
1.
2.
3.
4.
5.
On the Road Again - Willie Nelson
Running On Empty - Jackson Browne
Radar Love - Golden Earring
On the Road Again - Canned Heat
Ventura Highway - America
1.
2.
3.
4.
5.
1.
2.
3.
4.
5.
Top 18-24 year-old female Roadtrip tracks
1.
2.
3.
4.
5.
All Summer Long - Kid Rock
Beautiful Life - Union J
Superbad - Jesse McCarthy
Ride - JTR
Rockstar - A Great Big World
1.
2.
3.
4.
5.
Ride - JTR
1.
2.
3.
4.
5.
Ride - JTR
Top 55+ year-old male Sexy Time tracks
1.
2.
3.
4.
5.
Love Me - The Little Willies
Let’s Get it On - Marvin Gaye
A Whiter Shade of Pale - Procol Harum
Nobody Does it Better - Carly Simon
You’ll Never Find Another Love Like Mine - Lou Rawls
1.
2.
3.
4.
5.
1.
2.
3.
4.
5.
Top 18-24 year-old female Sexy Time tracks
1.
2.
3.
4.
5.
Love Faces - Trey Songz
Take You Down - Chris Brown
Dive In - Trey Songz
Birthday Sex - Jeremih
Kisses Down Low - Kelly Rowland
1.
2.
3.
4.
5.
1.
2.
3.
4.
5.
Top 55+ year-old male Break Up tracks
1.
2.
3.
4.
5.
Cry Like a Baby - The Box Tops
The Heart of the Matter - Don Henley
Crying in the Rain - The Everly Brothers
Crying - Roy Orbison
Always on my Mind - Willie Nelson
1.
2.
3.
4.
5.
1.
2.
3.
4.
5.
Top 18-24 year-old female Break Up tracks
1.
2.
3.
4.
5.
F*ck It (I Don’t Want You Back) - Eamon
Too Little, Too Late - JoJo
Potential BreakUp Song - Aly & AJ
Since U Been Gone - Kelly Clarkson
Leave (Get Out) - JoJo
Top 18-24 year-old female Break Up tracks
1.
2.
3.
4.
5.
F*ck It (I Don’t Want You Back) - Eamon
Too Little, Too Late - JoJo
Potential BreakUp Song - Aly & AJ
Since U Been Gone - Kelly Clarkson
Leave (Get Out) - JoJo
Experiment #5
Using scrubbing data to find the best part of a song
Using scrubbing data to find the best part of a song
Most common scrub-to location in the song based upon millions of listeners
Hotspot is an awesome
drop detector
What’s the most
prominent Hip Hop ?
Find the biggest drop by
an artist from the
19th Century
Classic Rock:
It’s all about the guitar solo
The greatest ‘moment’
in Classic Rock
The greatest scream
in Rock
The best 15 seconds of the
Rolling Stones
I will always love the chorus
Why do we care?
• 30 second previews generation
• The ultimate drop detector
• Guide automatic song transitions for Party
Mode and Run to the Beat
• “Play the best part” button
Just the Drop
Data about music
Music Metadata artists, albums songs, labels Cultural Data What people are saying about music Listener Data Who listened to what and when Acoustic Data What the music sounds like Acoustic Data How can we do this?
What the music sounds like Machine Listening
waveform
analysis
summarization
Audio
Tempo
Spectrum
Segment
Experiment #6
Is a human setting the beat for your favorite song?
Stewart Copeland
http://labs.echonest.com/click/
Stewart Copeland
Stewart Copeland
Stewart Copeland
Stewart Copeland
Machine score: 25% HUMAN Drummer
The Machine
The Machine
The Machine
The Machine
Machine score: 97% MACHINE Drummer
How we used to listen to music
How we listen to music now
How we listen to music now
Let’s bring back some of that interactivity in the next 8 experiments
Experiment #7
Find the most dramatic moments in your music
Finding the dramatic bits in your music
Experiment #8
Create advanced dynamic visualizations for your music
Experiment #9
Automatically remix your music
Data-driven algorithmic music remixing
First beat of every bar
track = remixer.analyze_track(’bad romance.mp3’)
beats = []
for beat in track['analysis']['beats']:
if beat['index_in_parent'] == 0:
beats.append(beat)
remixer.render(beats).export(’one romance.mp3’)
beats = []
beats.append(beat)
beats = []
beats.append(beat)
beats = []
beats.append(beat)
beats = []
beats.append(beat)
beat reversing
track = remixer.analyze_track('bad romance.mp3')
beats = track['analysis']['beats']
remixer.render(reversed(beats)).export('rev.mp3')
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16
beat reversing
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16
beat reversing
16 15 14 13 12 11 10
9
8
7
6
5
4
3
2
1
beat reversing
16 15 14 13 12 11 10
9
8
7
6
5
4
3
2
1
Experiment #10
‘Improve’ your favorite song by changing the drummer
The Bonhamizer
Some songs just need a little Bonzo
fun.
The Bonhamizer
fun.
The Bonhamizer
fun.
The Bonhamizer
fun.
The Bonhamizer
John Bonham
fun.
+
=
FUNNER!!
Experiment #11
Make your favorite song ‘swing’
Experiment #11
4 straight beats
Experiment #11
4 straight beats
4 swinging beats
Make any song ‘swing’
Experiment #12
Make your favorite song last forever
The Infinite Jukebox
For when your favorite song isn’t long enough
Experiment #13
Turn your favorite song into a canon
The Autocanonizer
Experiment #14
Play with your favorite song
Girl Talk in a Box
Treat your favorite song like a musical instrument
I’ve got 30 million songs in my pocket.
Now what?
Data that help us
better
understand the
world of music
will be
increasingly
important
Music + Data = Fun!
Some experiments in how we can use music data
to help improve the listening experience
Demos: bit.ly/MusicAtRailsConf
Paul Lamere
@plamere

PDF of RailsConf Spotify Keynote

Transcription

Similar documents

Live Earth Poster

CANDID McKINLEY

ROLE:!!JERRY!LEE!LEWIS!

Liner Notes Only - Dartmouth College

Ministries - North Pointe Community Church

Song: I Like The Way

w ith Miss Muffin

Cry Me a River