This was a nice demonstration of letting these robotic hands learn to play a keyboard. The technique is limited due to constraints of the hands, and is closer to parts of the technique of early keyboard instruments like an organ or a harpsichord, rather than that of a modern grand piano, which requires a lot more control of the body core, shoulders, elbows, arms, and wrists with the fingers doing as minimal a motion as possible. I suppose a similar algorithm learning to play on a grand piano using a full humanoid body could learn a technique that would be exciting to analyze.
Tangential but your comment has just given me a penny drop moment that I can isolate each degree of freedom (finger, wrist, elbow, shoulder) and still play a key to sound a note.
For context I have been working on playing multiple lines of music with right hand lately (Chopin etudes, much struggle...) and saw a video from a professional recommending playing the main melody "with the arm". This has helped a lot with the visualisation and now I can really have a cleaner technique playing the accompaniment above the escapement (using nice flat fingers) and the melody with a much richer tone (using the arm typically for the pinky).
I sort of understood in an intuitive nonverbal sense what I am doing but after your comment, the mechanics now makes a lot more sense of what I'm actually doing - thanks!
Glad it helps. Claudio Arrau thought of the fingers as “dead” with all the piano playing happening using larger muscles. It is a struggle to internalise this concept in some Chopin etudes (op 10, no 2 especially) but it almost immediately helps in others (op 25, no 1).
Yes, that's where this model is massively missing, an accomplished pianist plays with the whole arm, the elbow is crucial, it needs to stay very relaxed in order to accommodate the peculiarities of the human hand, the ring fingers and little fingers don't have as many tendons connecting and having a little sway in the elbows is crucial to making those fingers deliver the weight of the arm properly to the keybed. This model appears to ignore rotation of the wrist, and easing the elbows, where at a bare minimum it needs to start at the shoulder, and I'd argue the whole body.
Yeah but as gp pointed out that was the style du jour of baroque and early classical as the plucking keyboards were mostly played with a finger technique
This. In fact isolating and attempting to play purely from the fingers alone is a great way to develop RSI or any number of soft tissue injuries.
There's entire schools of thought around proper ergonomics with respect to piano playing - I took lessons in the Taubman technique which (very simplistically) tries to encourage movement farther upstream on the body.
We were able to make some improvements by tuning how the reward is distributed and also by first pretraining the agent on scales before fine-tuning them on the final pieces.
Thanks to Kevin Zakka for helping us get started with the RL environment!
Is this actually buildable? I'd like to hear it on a real piano because (IMHO) the one in the videos sounds bad.
I'd like to also hear how loud the mechanical noise of the machine playing the piano would be. Does the left hand work harder with the heavier keys? What would the hands be mounted to?
Yes. The robot used in this study is Dexterous Hand from Shadow Robot. It's a real product, and costs around $200-300k (for a single hand).
Controlling a real robot using a RL policy (model) trained in a simulation environment is also doable. It's called "Sim2Real" and has been widely experimented in the last decade (with mixed success, though).
Each key (on a good piano) is weighted to have the same force requirements to be played. Of course varying forces can be used to achieve different decibels, but the curve is the same across the piano.
> Every single key on a grand piano keyboard is weighted differently. This is because the strings for each note are slightly thinner and shorter in the treble register, becoming thicker and longer towards the bass register. As a result, there is greater resistance when playing low notes than when playing high notes. In other words, a heavier touch is required in the left hand and a lighter touch in the right hand. To emulate this in a digital piano, the keys are often individually weighted, with the lower keys heavier than the higher ones — something that’s called graded weighting.
It's not the strings though, the hammers are smaller as you go to the right in the higher register on the piano. The strings are slightly smaller in diameter, but remarkably shorter in length.
> We use the Musical Instrument Digital Interface (MIDI) standard to represent a musical piece as a sequence of time-stamped messages corresponding to "note-on" or "note-off" events. A message carries additional pieces of information such as the pitch of a note and its velocity.
I play the piano and I think MIDI does not have enough parameters to describe an acoustic piano. It's not just a single strike velocity that determines each sound. Where you play within the double escapement, how far down you hit the key, they all change the sound a bit.
That said though there are better MIDI synthesizers, e.g. https://www.modartt.com/
They still don't match an acoustic. I can tell the difference quite easily. I think a neural TTS retrained on piano data could do better.
Imediatelly reminded of the anonymous quote: "I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do laundry and dishes."
I understand it's a bad argument because 1) people may need assistance to do art/writing and 2) the advancements gained from teaching AI to do art can be applied to other non-artistic endeavors (e.g. piano AI could be really good to operate machines with lots of buttons and no computerized interface).
However, the cost cutting side of the argument is the one that bothers me because companies/people WILL use AI like that in place of actual humans because they're likely to be cheaper in the future. So that pianist or musician playing in a local restaurant can be sure their job will be automated away by a subpar AI and real humans will be relegated to very expensive locations (an extension of replacing humans with recorded music, in a way).
My pessismist side thinks greed will be the downfall of humanity.
The pianist playing in a restaurant was optimized by the player piano (which would only play if you put a nickel in) back in the late 1800s. Muzak put the final nail in that coffin in 1954.
The player piano technology comes from the barrel organ, which seems to have been on the first pipe organs.
I think if ones perspective of manual art changes, so does that of AI art, sorta like shifting priorities in life. Some priorities are higher than others. And past a certain point, AI art becomes a tempting option beneath a certain motivation level.
If for example you find manual art more fun to practice than AI art, you'd be less inclined to give AI companies money to generate art, since you could perhaps spend hours and hours practicing and have fun.
And of course if you don't find manual art enjoyable, but still don't want to give up on the "art" idea entirely, AI art crosses the threshold into enjoyability, so the end product is at least in front of your eyes.
As for how to learn how to make one type of art more fun than the other, if ones preferences are set in stone, it's pretty tough. The AI/non-art-inclined crowd might invert your quote so the laundry and dishes get prioritized, because different people have different priorities.
I agree, but live performance in the age of recording is already a luxury good, and it is unlikely an AI would replace live performance any more than a player piano or CD player would (or has.)
I can't see how in a world full of recordings how in person, live performance art could get anything but vastly more valuable.
I saw a stage performance of Clue a few weeks ago. It was lovely. Nothing was AI generated, and couldn't be, because it was people physically making art live. How nice.
I will continue to pay for those experiences and to prioritize them over recorded ones.
People want robots do (low-value && computationally intensive) tasks - I think it's fair and okay by now to say unfolding wet clumps came out of a washing machine belongs to a class of tasks that are computationally intensive for human brains.
Your comment reminded me of a scene from the movie "El Mariachi". In it, the title character walks into a bar looking for a job as a Mariachi, and is faced with the bar owner telling him something like "Why do I need a Mariachi when I can have a full band?", then pointing to a guy resting in the corner next to an organ (an "electronic organ", of the family type, as opposed to, you know, a hammond or something like that). The resting guy presses the "demo" button and the organ plays Mariachi music on its own ...
I share your pessimism partially, but I also wonder: if the system keeps optimizing for profit, at some point there just won't be enough people in the world to sell products and services to. I don't know what the answer is.
> UC Berkeley, Google DeepMind, Stanford University
I've personally experienced how research around this time was being shut down because of AI doomerism. People were getting laid off because of it. It's clear to me that these institutions actively spread AI doomerism so that they have full control over it. They actively called for a stop in AI research so that their personal labs can leap forward ahead. It was a little too on the nose for Big Tech, but they don't understand nuance.
If you know the precise state of each motor in a robot hand, you can compute the absolute position of its fingertip, right?
To make a robot play piano, you need to solve the reverse problem of it i.e. to compute the motor states from the target robot position. This problem is called "Inverse Kinematics" and not fully resolved to this day.
Reinforcement Learning (RL) people claim that, if you let a robot move randomly, and give it apporopriate rewards depending on the resulting state, eventually it learns to solve the problem by itself (just like a human baby eventually learns how to walk through trial & error).
Now, you are looking at the state of the art of RL in 2023.
No, a state-based RL research like this is essentially an IK problem. Given a goal position in the world frame, you need to find out the motor configuration to move your EEF to that goal position.
> IK to my knowledge is well known in every setting I am aware of.
Really? When I was working in this field, I've actually never seen anyone who used numerical/analytical IK methods on real robots.
Granted that I was not a Robotics engineer (I was a Deep Learning engineer in the team), but my impression at the time was that no practical IK solution was available for a robot with 8 DOF, let alone a 20 DOF robot like Shadow Hand.
The idea is that the motor control is pretty hard to program, and by using reinforcement learning, one only has to tell the computer whether it hits the right keys (reward).
The reinforcement learning algorithm takes in the state of the fingers and piano keys, and outputs the required motor actions to perform well.
Also, as you are too lazy to read an article but still feel entitled to comment on it, I herby give you -1 reward. Let's see if that helps to improve your character.
This was a nice demonstration of letting these robotic hands learn to play a keyboard. The technique is limited due to constraints of the hands, and is closer to parts of the technique of early keyboard instruments like an organ or a harpsichord, rather than that of a modern grand piano, which requires a lot more control of the body core, shoulders, elbows, arms, and wrists with the fingers doing as minimal a motion as possible. I suppose a similar algorithm learning to play on a grand piano using a full humanoid body could learn a technique that would be exciting to analyze.
Tangential but your comment has just given me a penny drop moment that I can isolate each degree of freedom (finger, wrist, elbow, shoulder) and still play a key to sound a note.
For context I have been working on playing multiple lines of music with right hand lately (Chopin etudes, much struggle...) and saw a video from a professional recommending playing the main melody "with the arm". This has helped a lot with the visualisation and now I can really have a cleaner technique playing the accompaniment above the escapement (using nice flat fingers) and the melody with a much richer tone (using the arm typically for the pinky).
I sort of understood in an intuitive nonverbal sense what I am doing but after your comment, the mechanics now makes a lot more sense of what I'm actually doing - thanks!
Glad it helps. Claudio Arrau thought of the fingers as “dead” with all the piano playing happening using larger muscles. It is a struggle to internalise this concept in some Chopin etudes (op 10, no 2 especially) but it almost immediately helps in others (op 25, no 1).
Interesting, I've read similar things about calligraphy/writing (from the elbow!), hadn't connected it to piano playing.
Yes, that's where this model is massively missing, an accomplished pianist plays with the whole arm, the elbow is crucial, it needs to stay very relaxed in order to accommodate the peculiarities of the human hand, the ring fingers and little fingers don't have as many tendons connecting and having a little sway in the elbows is crucial to making those fingers deliver the weight of the arm properly to the keybed. This model appears to ignore rotation of the wrist, and easing the elbows, where at a bare minimum it needs to start at the shoulder, and I'd argue the whole body.
Yeah but as gp pointed out that was the style du jour of baroque and early classical as the plucking keyboards were mostly played with a finger technique
This. In fact isolating and attempting to play purely from the fingers alone is a great way to develop RSI or any number of soft tissue injuries.
There's entire schools of thought around proper ergonomics with respect to piano playing - I took lessons in the Taubman technique which (very simplistically) tries to encourage movement farther upstream on the body.
https://www.ednagolandsky.com/the-taubman-approach-basic-pri...
Any way we can get the Westworld intro song into the demo list?
This was the basis of a project I did for my deep reinforcement learning class!
https://ericye16.com/stanford-cs224r
We were able to make some improvements by tuning how the reward is distributed and also by first pretraining the agent on scales before fine-tuning them on the final pieces.
Thanks to Kevin Zakka for helping us get started with the RL environment!
did you guys ever try having the agents play the song slower at first?
Is this actually buildable? I'd like to hear it on a real piano because (IMHO) the one in the videos sounds bad.
I'd like to also hear how loud the mechanical noise of the machine playing the piano would be. Does the left hand work harder with the heavier keys? What would the hands be mounted to?
> Is this actually buildable
Yes. The robot used in this study is Dexterous Hand from Shadow Robot. It's a real product, and costs around $200-300k (for a single hand).
Controlling a real robot using a RL policy (model) trained in a simulation environment is also doable. It's called "Sim2Real" and has been widely experimented in the last decade (with mixed success, though).
Thanks for that information.
There's a video that shows the hand moving and it doesn't look like it would be fast enough for anything but relatively slow songs:
https://youtu.be/2ggpvigfEZE?t=55
This is more recent, and arguably going after something different(fingertip sensation), but I'm sure has overlap as well:
https://www.youtube.com/watch?v=9AofrJJaS8U
What do you mean “heavier keys”?
Each key (on a good piano) is weighted to have the same force requirements to be played. Of course varying forces can be used to achieve different decibels, but the curve is the same across the piano.
Probably this:
> Every single key on a grand piano keyboard is weighted differently. This is because the strings for each note are slightly thinner and shorter in the treble register, becoming thicker and longer towards the bass register. As a result, there is greater resistance when playing low notes than when playing high notes. In other words, a heavier touch is required in the left hand and a lighter touch in the right hand. To emulate this in a digital piano, the keys are often individually weighted, with the lower keys heavier than the higher ones — something that’s called graded weighting.
https://hub.yamaha.com/pianos/p-digital/a-quick-guide-to-wei...
It's not the strings though, the hammers are smaller as you go to the right in the higher register on the piano. The strings are slightly smaller in diameter, but remarkably shorter in length.
> We use the Musical Instrument Digital Interface (MIDI) standard to represent a musical piece as a sequence of time-stamped messages corresponding to "note-on" or "note-off" events. A message carries additional pieces of information such as the pitch of a note and its velocity.
I play the piano and I think MIDI does not have enough parameters to describe an acoustic piano. It's not just a single strike velocity that determines each sound. Where you play within the double escapement, how far down you hit the key, they all change the sound a bit.
That said though there are better MIDI synthesizers, e.g. https://www.modartt.com/ They still don't match an acoustic. I can tell the difference quite easily. I think a neural TTS retrained on piano data could do better.
Nice work! And great interactive 3d application. My 6yo had a lot of fun annoying the robot while it's playing by forcefully moving its hands around.
All the right notes but not necessarily at the right time.
Which is exactly the wrong way around from what's expected for a pianist :)
Has anyone tried to get a robot to do oil paintings?
This company: https://www.acrylicrobotics.ca/
Funny enough I’m listening to Rhapsody in Blue while browsing HN. I’d like to see it do that for 17 minutes.
waiting for Robot Devil.
Imediatelly reminded of the anonymous quote: "I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do laundry and dishes."
I understand it's a bad argument because 1) people may need assistance to do art/writing and 2) the advancements gained from teaching AI to do art can be applied to other non-artistic endeavors (e.g. piano AI could be really good to operate machines with lots of buttons and no computerized interface).
However, the cost cutting side of the argument is the one that bothers me because companies/people WILL use AI like that in place of actual humans because they're likely to be cheaper in the future. So that pianist or musician playing in a local restaurant can be sure their job will be automated away by a subpar AI and real humans will be relegated to very expensive locations (an extension of replacing humans with recorded music, in a way).
My pessismist side thinks greed will be the downfall of humanity.
The pianist playing in a restaurant was optimized by the player piano (which would only play if you put a nickel in) back in the late 1800s. Muzak put the final nail in that coffin in 1954.
The player piano technology comes from the barrel organ, which seems to have been on the first pipe organs.
I think if ones perspective of manual art changes, so does that of AI art, sorta like shifting priorities in life. Some priorities are higher than others. And past a certain point, AI art becomes a tempting option beneath a certain motivation level.
If for example you find manual art more fun to practice than AI art, you'd be less inclined to give AI companies money to generate art, since you could perhaps spend hours and hours practicing and have fun.
And of course if you don't find manual art enjoyable, but still don't want to give up on the "art" idea entirely, AI art crosses the threshold into enjoyability, so the end product is at least in front of your eyes.
As for how to learn how to make one type of art more fun than the other, if ones preferences are set in stone, it's pretty tough. The AI/non-art-inclined crowd might invert your quote so the laundry and dishes get prioritized, because different people have different priorities.
I agree, but live performance in the age of recording is already a luxury good, and it is unlikely an AI would replace live performance any more than a player piano or CD player would (or has.)
I can't see how in a world full of recordings how in person, live performance art could get anything but vastly more valuable.
I saw a stage performance of Clue a few weeks ago. It was lovely. Nothing was AI generated, and couldn't be, because it was people physically making art live. How nice.
I will continue to pay for those experiences and to prioritize them over recorded ones.
No doubt it'll be more valuable but the number of opportunities (to watch and play at those) will decrease globally.
People want robots do (low-value && computationally intensive) tasks - I think it's fair and okay by now to say unfolding wet clumps came out of a washing machine belongs to a class of tasks that are computationally intensive for human brains.
Your comment reminded me of a scene from the movie "El Mariachi". In it, the title character walks into a bar looking for a job as a Mariachi, and is faced with the bar owner telling him something like "Why do I need a Mariachi when I can have a full band?", then pointing to a guy resting in the corner next to an organ (an "electronic organ", of the family type, as opposed to, you know, a hammond or something like that). The resting guy presses the "demo" button and the organ plays Mariachi music on its own ...
I share your pessimism partially, but I also wonder: if the system keeps optimizing for profit, at some point there just won't be enough people in the world to sell products and services to. I don't know what the answer is.
Edit: typo
The only real answer here.
the fact that the failure modes come from limitations of the robotic hands is super interesting
Wasted opportunity to not play the Westworld Theme
> UC Berkeley, Google DeepMind, Stanford University
I've personally experienced how research around this time was being shut down because of AI doomerism. People were getting laid off because of it. It's clear to me that these institutions actively spread AI doomerism so that they have full control over it. They actively called for a stop in AI research so that their personal labs can leap forward ahead. It was a little too on the nose for Big Tech, but they don't understand nuance.
This's awesome!!!
Honestly,
this is really bad. It might be a breakthrough of what you are doing, but when I listen to the output all of the timing and phrasing is aweful.
Alright then, post your ML simulation of a piano player, I'll wait :)
[flagged]
I mean, keys are always in the place where you expect them to be, so what is the challenge here? (Yes, too lazy to read the article).
> what is the challenge here?
If you know the precise state of each motor in a robot hand, you can compute the absolute position of its fingertip, right?
To make a robot play piano, you need to solve the reverse problem of it i.e. to compute the motor states from the target robot position. This problem is called "Inverse Kinematics" and not fully resolved to this day.
Reinforcement Learning (RL) people claim that, if you let a robot move randomly, and give it apporopriate rewards depending on the resulting state, eventually it learns to solve the problem by itself (just like a human baby eventually learns how to walk through trial & error).
Now, you are looking at the state of the art of RL in 2023.
The challenge here is not IK. Sure, you don’t need IK if your policy directly operates in joint space but this isn’t the motivation for doing RL.
IK to my knowledge is well known in every setting I am aware of.
> The challenge here is not IK.
No, a state-based RL research like this is essentially an IK problem. Given a goal position in the world frame, you need to find out the motor configuration to move your EEF to that goal position.
> IK to my knowledge is well known in every setting I am aware of.
Really? When I was working in this field, I've actually never seen anyone who used numerical/analytical IK methods on real robots.
Granted that I was not a Robotics engineer (I was a Deep Learning engineer in the team), but my impression at the time was that no practical IK solution was available for a robot with 8 DOF, let alone a 20 DOF robot like Shadow Hand.
The idea is that the motor control is pretty hard to program, and by using reinforcement learning, one only has to tell the computer whether it hits the right keys (reward).
The reinforcement learning algorithm takes in the state of the fingers and piano keys, and outputs the required motor actions to perform well.
Also, as you are too lazy to read an article but still feel entitled to comment on it, I herby give you -1 reward. Let's see if that helps to improve your character.
Where's yours? That's probably the best way to find all the challenges.
I think the robot learnt to play by itself, right?
"Player Piano"
The thing needs a teacher, terrible technique! /s
It plays almost as badly as I do.
I think this is worth mentioning for people who don't play piano. It's technique is terrible and bizarre. It's flicking the keys with its fingernails.
[dead]
[dead]