> Resilient & creative robots

We develop new algorithms for creative adaptation in robotics.

If we want robots that can operate for a very long time, then we have to take into account that we can't anticipate all the possible problems. We have to give the robots the ability to cope with all these problems by themselves.

To do this, we need two ingredients. First, we need a very creative process: if the robot needs to cope with crazy unforeseen situations, then it must be able to come with crazy solutions. Second, we need a reasonably fast process, because if a robot need one month to deal with a broken leg, then the robot cannot work during this period.

Evolutionary algorithms are good candidates for this creative process. However, they require too many tests on the robot to be used in real scenarios: Evolution in nature took millions of years to perfect the lifeforms of the Earth; our robots obviously don't have this time to recover from an unexpected failure, or to understand that the ground is now covered of vulcanic dust.

We could use a comptuter simulation to accelerate the evolutionary process, but updating a simulation "on the fly" is a complex and slow process.

Our breakthrough came when we were working on the transferability approach. We realized that we don't need to update the simulation to be able to use it. We only need to be able to avoid the parts of the simulations that have become inaccurate, that is behaviors that work differently in the simulation and on the real robot. In our robot, this means that the embedded simulation of itself -- it's "self-model" -- sometimes matches the reality, and sometimes not. If we know how to focus the search on the behaviors that are accurately simulated, then we can use the simulation.

Let's think about a legged robot with a broken leg. All the gaits the that use the broken leg will have a very different result in simulation and in reality, but the gaits that don't use this broken leg will result in a similar behavior in reality and in simulation. We therefore designed an algorithm that learns to predict what behaviors will work "as expected" and "not as expected".

Once the robot can predict the movements that have to be avoided, it uses the simulation to search for a new behavior. Our robot never identifies the problem. It never knows that the leg is broken. It only develops an "intuition" of what it can and cannot use. Wecall this new algorithm the "T-resilience" (for "transferability-based resilience").

This new learning process is much faster than all the previous attempts to learn new behaviors after a failure. Now we can use evolutionary algorithms in the real world, with real robots, in situations that truly require to discover a new behavior.

We tested our approach one of our hexapod robot (see the video). The robot has to discover by itself new gaits when we remove a leg, when we break a leg (we took a saw and removed half of the leg) and when we disconnect some motors. The robot is equipped with a Kinect-like sensor (which is actually a 3D camera) so that it can assess how well it walks using only internal sensors.

In all these situations, in less than 20 minutes and only 25 tests on the robot (instead of thousands with previous approaches), the robot is able to walk again. Moreover, each test lasts only 3 seconds, therefore the vast majority of time is spent in computations. With faster computers, the algorithm will be faster. Reality cannot be sped up, but we discovered that we only need to find the right link between reality and simulation to use the simulation. And now we can harness the power of evolution to power real robots.

Technically, our algorithm relies on the transferability approach




Publications

  • S. Koos, A. Cully and J.-B. Mouret. Fast Damage Recovery in Robotics with the T-Resilience Algorithm, International Journal of Robotics Research. 2013. [pdf]
  • A. Cully and J.-B. Mouret. Behavior Repertoire Learning in Robotics, 2013. Proceedings of GECCO. [pdf]

Funding

Funding provided by the ANR (project Creadapt, ANR-12-JS03-0009)

Participants

  • Jean-Baptiste Mouret (PI)
  • Antoine Cully (ph.d)
  • Sylvain Koos (post-doc)
  • Stephane Doncieux
  • Christophe Grand
  •