Last weekend I went to this music hack day in Boston locating in NERD (microsoft New England Research and Development center) close to MIT right across the Charles River. Absolutely beautiful place to work.
Attended several talks which the topics are basically those start-ups who would like people to use their APIs to build stuff. Anyway, pp and I tried Echonest's API's which have two major functions. One is to analyze music using signal processing and machine learning techniques (similar to what I am doing) and the other is text searching for all music related metadata including blogs, comments, etc.
What we did? We took pp's musicpainter which is a graphical collaborative composing interface and align the canvas grid with any music using Echonest's APIs to get tempo, beats and key. Then one can play or compose along with the music you like. (sort of) It is been great fun working with pp. I had a good time there. Lots of free food in the office and delicious home made dishes at pp's. So warm!
Monday, November 30, 2009
Monday, November 23, 2009
Hard to define
The definition or I should say the composition of music is the interaction between silence and sound, just like an image consists of light and shadow...
Saturday, November 14, 2009
Envisioning Information
Recently skimmed through this book "Envisioning Information" written by Edward R. Tufte. Though it is published in 1990, the concept of visualization is kinda useful.
I did not read closely page by page, basically I just look at the graphs and try to get the idea of what he is trying to explain. Not bad! Below I would like to list the chapter titles of this book, which are the main concept of thinking about visualization I think.
Micro/Macro Readings
Layering and Separation
Small Multiples
Color and Information
Narratives of Space and Time
Definitely I will come back and read through the whole book in detail. But for now I just wanna get some idea whirling in my head when I am searching for ideas.
I did not read closely page by page, basically I just look at the graphs and try to get the idea of what he is trying to explain. Not bad! Below I would like to list the chapter titles of this book, which are the main concept of thinking about visualization I think.
Micro/Macro Readings
Layering and Separation
Small Multiples
Color and Information
Narratives of Space and Time
Definitely I will come back and read through the whole book in detail. But for now I just wanna get some idea whirling in my head when I am searching for ideas.
Friday, November 13, 2009
Remembering names
I know that social ability is very crucial here and I am trying to develop is little by little. Recently I found that there are some name issues hard to deal with.
Of course you need to remember one's name before any social ability to be repeated. For me it is always the case: "Hey my name is xxx." "Nice to meet you!" This is usually the starting point and I usually do not get the name at the first place unless it is very common one (especially in US there are so many weird name including mine everywhere...). Rarely I asked again and try to write down and memorize but usually I just let it go and when the next time we met, it is a little bit embarrassing. I am pretty sure that the other guy feels the same because my name is definitely more difficult. This is the first and big obstacle to me then.
Anyway, trying to figure out a way to remember more names... Maybe just need to pay more attention when a name is presented...
Of course you need to remember one's name before any social ability to be repeated. For me it is always the case: "Hey my name is xxx." "Nice to meet you!" This is usually the starting point and I usually do not get the name at the first place unless it is very common one (especially in US there are so many weird name including mine everywhere...). Rarely I asked again and try to write down and memorize but usually I just let it go and when the next time we met, it is a little bit embarrassing. I am pretty sure that the other guy feels the same because my name is definitely more difficult. This is the first and big obstacle to me then.
Anyway, trying to figure out a way to remember more names... Maybe just need to pay more attention when a name is presented...
Thursday, November 12, 2009
IDEA: Music shaker
Just read a paper from ismir 2009 that several music snippets are remixed according to their tonal and tempo information. I mean, I have this idea very long time ago... But I did not do anything and now it is others' contribution. Though I am not saying I will be able to solve the question, I am just lazy and think way much more than work.
Anyway, if the technology is already there, what else can we do? Or can we go beyond this? Therefore the application idea pop out, simply an iphone app. If we can pick two songs and shake the iphone which indicates that blend/mix this two songs together in "a way". Can be if shake harder, they will blend better like phrase by phrase blend or something. And the technology required are tempo estimation as well as key detection with phase vocoder to synchronize these two attributes.
Anyway, if the technology is already there, what else can we do? Or can we go beyond this? Therefore the application idea pop out, simply an iphone app. If we can pick two songs and shake the iphone which indicates that blend/mix this two songs together in "a way". Can be if shake harder, they will blend better like phrase by phrase blend or something. And the technology required are tempo estimation as well as key detection with phase vocoder to synchronize these two attributes.
Arc diagram
This is a current project I have been working for a while. Check out Shape of Song, Martin Wattenberg created this way to visualize a song, but from midi data, which you know exactly the notes. What I did is using the similar arc diagram on real audio data, to be specific, on the feature extracted from real audio data.
From the plot, basically the structure of the song is described. The idea of this arc diagram is that when those two points are identical (highly similar) you draw an arc line to connect those two dots. So when there are repeated patterns, they will be displayed. I am still exploring different features to be used while the same time trying to figure out better way to put the information in this plot. BTW, this song is a classical piano song "Fur Elise" (aka the trash truck song in Taiwan if I did not mix match).
IDEA: Piano paint
Oh well, I know the name is kinda lame. Anyway, cause the idea is vague too. Just a quick note about an idea come up to my mind the other day.
What if we can paint by playing instrument? Try to narrow down, the first experiment will be build a system which takes the input of a keyboard as midi signal and then plot something on to the screen. I have a feeling that this is gonna be fun and a pretty cool interactive art. Though I have not put much thought to it yet.
Below are some quick memos:
Randomly pick the starting point for each music sentence. According to the note and harmony you played, different color can be chosen and the size of the stroke and touch can be related to how hard you play the keyboard (loudness). Like I can imagine that if some weird chord series are played loudly, there will be some explosive color objects appeared on the screen.
I am sure that this idea must be down by someone already. But I would love to build one too. Maybe when I have time, can start doing some literature search first (google...). And I am thinking about taking this as a chance to learn max/msp.
What if we can paint by playing instrument? Try to narrow down, the first experiment will be build a system which takes the input of a keyboard as midi signal and then plot something on to the screen. I have a feeling that this is gonna be fun and a pretty cool interactive art. Though I have not put much thought to it yet.
Below are some quick memos:
Randomly pick the starting point for each music sentence. According to the note and harmony you played, different color can be chosen and the size of the stroke and touch can be related to how hard you play the keyboard (loudness). Like I can imagine that if some weird chord series are played loudly, there will be some explosive color objects appeared on the screen.
I am sure that this idea must be down by someone already. But I would love to build one too. Maybe when I have time, can start doing some literature search first (google...). And I am thinking about taking this as a chance to learn max/msp.
Visualization
Recently I am pretty into information/data visualization. The idea is to explore some ways to visualize music in different levels. Level here means from within a song level to further the whole music collection level.
So the idea for visualization is to show something human might not be able to perceive if they are described in the old ways. For example, when you are listening to music, it is harder to tell the structure of the music (verse vs chorus, how many times the chorus got repeated, so on and so forth). However, those information can be easily shown by a simple 2-D plot. This is just one application and we believe that there are much more possibilities out there for exploration. If the plot does not mean anything, at least it needs to be beautiful and pleasant to see.
Just had a meeting with my adviser who also has an engineering background and claim that he does not really know about aesthetic. Oh well, neither do I... I just started to read the book "Visualizing Data" by Ben Fry, a guy from MIT media lab who did his PhD work in "Computational Information Design" and also the co-inventor of a well known visualization sketch tool called "Processing". I do not have much to discuss about the book yet, so far I can see that the whole process of visualizing data is laid out in the book. And I have played with processing a bit, pretty friendly and intuitive programming language.
Good start though, will see how far I can go, probably will go take one to two classes about visualization and graphical design next semester in order to learn the thinking process of a designer. Looking forward to it!
So the idea for visualization is to show something human might not be able to perceive if they are described in the old ways. For example, when you are listening to music, it is harder to tell the structure of the music (verse vs chorus, how many times the chorus got repeated, so on and so forth). However, those information can be easily shown by a simple 2-D plot. This is just one application and we believe that there are much more possibilities out there for exploration. If the plot does not mean anything, at least it needs to be beautiful and pleasant to see.
Just had a meeting with my adviser who also has an engineering background and claim that he does not really know about aesthetic. Oh well, neither do I... I just started to read the book "Visualizing Data" by Ben Fry, a guy from MIT media lab who did his PhD work in "Computational Information Design" and also the co-inventor of a well known visualization sketch tool called "Processing". I do not have much to discuss about the book yet, so far I can see that the whole process of visualizing data is laid out in the book. And I have played with processing a bit, pretty friendly and intuitive programming language.
Good start though, will see how far I can go, probably will go take one to two classes about visualization and graphical design next semester in order to learn the thinking process of a designer. Looking forward to it!
The new role of this blog
I am going to add one more function to this blog here, that I would like to write down things related to my research including thoughts, idea, memos, etc. Just try to catch some transient memory and see what I can really achieve.
Subscribe to:
Posts (Atom)