Giorgio Metta

The RobotCub project
Giorgio Metta (just presenting)
and many contributors
Cognitive Humanoids Laboratory
Dept. of Robotics, Brain and Cognitive Science
Italian Institute of Technology
Collaborations
the RobotCub Consortium
U. of Genoa/IIT
Scuola S. Anna
U. of Zurich
U. of Uppsala
U. of Ferrara
U. Hertfordshire
IST - Lisbon
U. Sheffield/IIT
EPFL
Telerobot S.r.l
RobotCub goals
Design a humanoid robot platform, namely
the iCub
Make it the platform of choice for
researchers in artificial cognitive systems
Study cognition from a developmental
perspective (neuroscience)
A scientific approach to robotics
(the geeks’ revenge!)
•
By providing the iCub to the community, we offer:
– Clear reproducible experiments: papers advertize research, but the
science stays in the code. Code is published openly and every iCub
owner can use this code to repeat experiments;
– A move from proof by video into proof by experimental evidence, which
is progress for artificial cognitive systems and robotics: it’s not just a
matter of calling us “robotic scientists”, it’s about making robotics into a
science;
– Code quality via natural selection: Code that works poorly tends to die
off and consequently “nice” methods that are irrelevant in practice will
not survive;
– Geeks’ revenge: implementation, timing, efficiency does matter:
“muscles” aren’t only shown on mathematical skills but on
practical issues as well;
– A cultural experiments: fusing neuroscience and engineering
creates a new culture where new ideas thrive.
Month 0
Month 12
Month 24
Month 36
Month 48
Month 65
Division of labor
UNISAL, Sheffield, IIT
IIT, UGDIST, TLR
IST
2004
SSSA
UGDIST
2005
2006
2007 2008 2009
The iCub
The iCub: quick summary
The iCub is the humanoid baby-robot
designed as part of the RobotCub
project
– The iCub is a full humanoid
robot sized as a three and half
year-old child
– The total height is 104cm
– It has 53 degrees of freedom,
including articulated hands to
be used for manipulation and
gesturing
– The robot can crawl and sit and
autonomously transition from
crawling to sitting and viceversa
– The robot is GPL/FDL:
software, hardware, drawings,
documentation, etc.
iCub features
• Hands: we started the
design of the iCub from
the hands
– 5 fingers, 9 degrees of
freedom, 19 joints
• Sensors: human-like
sensorization (e.g. no lasers)
– Cameras, microphones, gyros,
accelerometers, encoders,
force/torque, tactile…
• Electronics: flexibility for
reseach
– Custom electronics, small,
programmable (DSP)
6-axis force/torque sensor
• Semiconductor strain
gauges
• On board signal
conditioning, sampling,
and calibration
• Digital output: CAN bus
Design: Nikos Tsagarakis
Electronics: Claudio Lorini
Promoting the iCub
•
RobotCub Open Call
– 31 participants, 7 winners received a copy of the iCub free of charge
– UPMC Paris, Imperial London, Inserm Lyon, TU Munich, METU Ankara,
Pompeu Fabra Barcelona, Urbana-Champaign USA, IST Lisbon, EPFL
Lausanne
•
Further development…
–
–
–
–
•
EU project ITALK: 4 iCub’s have been built
EU project ImClever: 3 iCub’s will be built (mid 2010)
EU project RoboSkin: a skin system compatible with iCub
EU project CHRIS: safety features for the iCub
Collaborations
– University of Karlsruhe: new and longer legs
•
Simulator:
– Open Source simulator based on ODE/Newton and as a model in
Webots
Wiki
SVN
Part lists
Drawings
The iCub Software
The iCub software philosophy
•
•
•
•
•
•
One processor is never enough
Modularity
Minimal interference
Stopping hurts
Humble approach (thin middleware)
Exploit diversity
YARP: the plumbing
• Designed to increase code sharing and longevity
• YARP is an Open Source software library for
humanoid robotics
• Started in 2002 as an independent project, has been
adopted by the consortium as the middleware for the
iCub platform
• Major overhaul led to version 2
Exploit diversity: portability
• Operating system portability:
– Adaptive Communication Environment
C++ OS wrapper
e.g. threads, semaphores, sockets
• Development environment: CMake
LINUX:
Makefiles,
Kdevelop
files, ...
Project
description
(.txt)
C/C++
library
C/C++
library
WINDOWS:
MSVC files,
Borland files,
...
C/C++
library
OSX:
Makefiles,
Xcode files,
...
C/C++
library
Modularity
machine 2
motor_control
machine1
tracker
/tracker/position
/motor/position
tcp
machine 3
yarpview
yarpdev
/camera
/tracker/image
mcast
/viewer2
udp
yarpview
mcast
• Connections can use different protocols
• Ports belong to processes
• Processes can be on different machines/OS
/viewer1
More on connections
•
•
•
•
peer-to-peer
synchronous/asynchronous
different buffering policies
protocols:
–
–
–
–
–
–
local (within process)
shared memory (same machine)
tcp, tcp_fast (without ack)
udp
mcast
variants (e.g. binary/text/html)
Physical Network
Example: RobotCub
Gigabit Ethernet (with tcp, udp, multicast traffic)
Desktops
(Linux/Windows)
blade cluster
(Linux)
pc104
(Linux/
Windows)
YARP development
some numbers
YARP is “stable”, size did not increase significantly during
the past few years
(source: www.ohloh.net)
start of RobotCub
today
iCub repository: evolution
We started by favoring
commits of code in
almost any form
A few modules: Meeting
in Genoa, October 2006,
CVS repository just
established
Today – January 2010
120
modules
applications
100
80
60
40
20
0
2004
2005
2006
2008
2009
~120 modules
~ 15 applications
 1M(!!) lines of code
list continues…
The iCub repository on SourceForge
(SVN)
• easier to maintain
• some tools for free (ML, download page, statistics)
• better visibility
project starts
repository moved to
SourceForge.net,
and subversion
The “manual”
www.icub.org
•
•
•
•
Installation instructions
List of dependencies
More tutorials
Documentation and coding guidelines
Dependencies
• Build a list of “allowed” dependencies
• Provide support for a given set of libraries
(instructions for installation, repository)
• Verify dependencies on automatic test
machines
www.icub.org  Software  Compile Status
Testing on:
• Windows
• Linux: Debian
• Linux: latest Ubuntu
Documentation
•
•
•
•
Doxygen
Provided templates for documenting modules and applications
Documentation is automatically parsed and posted online
Work to maintain the documentation and keep it accessible
www.icub.org  software
Examples
Simple example: a countdown
#include <yarp/os/all.h>
#include <stdio.h>
using namespace yarp::os;
#include <yarp/os/all.h>
#include <stdio.h>
using namespace yarp::os;
int main(int argc, char *argv[]) {
if (argc!=2) return 1;
Network::init();
int main(int argc, char *argv[]) {
if (argc!=2) return 1;
Network::init();
BufferedPort<Bottle> out;
out.open(argv[1]);
BufferedPort<Bottle> in;
in.open(argv[1]);
for (int i=10; i>=0; i--) {
printf("at %d\n", i);
Bottle& msg = out.prepare();
msg.clear();
msg.addString("countdown");
msg.addInt(i);
out.write();
Time::delay(1);
}
int count = 1;
while (count>0) {
Bottle *msg = in.read();
count = msg->get(1).asInt();
printf("at %d\n", count);
}
Network::fini();
return 0;
}
make_count.cpp
Network::fini();
return 0;
}
view_count.cpp
Simple example: a countdown
• make_count generates a countdown message
– countdown 10
– countdown 9
– ...
• view_count views a countdown message
• Neither program mentions the other
• Neither program dictates how the message will
be transmitted
Simple example: a countdown
$ ./make_count /count
yarp: Port /count listening
at 10
at 9
...
$
C:\> view_count /view
yarp: Port /view listening
YARP Network
make_count /count
/count
view_count /view
/view
Simple example: a countdown
$ ./make_count /count
yarp: Port /count listening
at 10
at 9
...
$
C:\> view_count /view
yarp: Port /view listening
Simple example: a countdown
$ ./make_count /count
yarp: Port /count listening
at 10
at 9
yarp: output /count to /view, tcp
at 8
at 7
..
$ yarp connect /count /view
Added output: /count to /view, tcp
$
C:\> view_count /view
yarp: Port /view listening
yarp: Input /count to /view, tcp
at 8
at 7
...
YARP Network
make_count /count
/count
view_count /view
tcp
/view
Simple example: a
countdown
$ ./make_count /count
yarp: Port /count listening
at 10
at 9
yarp: output /count to /view, tcp
at 8
at 7
..
$ yarp connect /count /view
Added output: /count to /view, tcp
$
C:\> view_count /view
yarp: Port /view listening
yarp: Input /count to /view, tcp
at 8
at 7
...
Simple example: a countdown
$ ./make_count /count
yarp: Port /count listening
at 10
at 9
yarp: output /count to /view, tcp
at 8
at 7
yarp: output /count to /view2, udp
at 6
...
C:\> view_count /view
yarp: Port /view listening
yarp: Input /count to /view, tcp
at 8
at 7
at 6
at 5
...
$ yarp connect /count /view
Added output: /count to /view, tcp
$ view_count /view2
yarp: Port /view2 listening
yarp: Input /count to /view2, udp
at 6
at 5
...
$ yarp connect /count /view2 udp
Added output: /count to /view, udp
YARP Network
make_count /count
/count
view_count /view
tcp
/view
view_count /view2
udp
/view2
YARP Devices
• There are three separate concerns related to devices
in YARP:
– Implementing specific drivers for particular devices
– Defining interfaces for device families
– Implementing network wrappers for interfaces
• Basic idea: if you view your devices through well
thought out interfaces, the impact of device change
can be minimized.
A Light Touch
• New devices come out all the time – needs to be
easy to connect them to existing code
• YARP needs a minimal “wrapper” class to match
vendor-supplied library with relevant interfaces
that capture common capabilities
• YARP encourages separating configuration from
source code – separating the “plumbing”
• Devices and communications remain distinct
concerns
Example: New WebCam
Super-WebCam
device
Super-WebCam
Vendor-Supplied
Library/Code
Example: New WebCam
Super-WebCam
device
YARP Device Interface
Audio Interface
Image Interface
Super-WebCam
Vendor-Supplied
Library/Code
Misc
Example: New WebCam
Super-WebCam
device
YARP Device Interface
Audio Interface
Image Interface
Super-WebCam
Vendor-Supplied
Library/Code
Misc
Reusable code
Example: Old Framegrabber
analogue
framegrabber
YARP Device Interface
Image Interface
analogue
framegrabber
Vendor-Supplied
Library/Code
Misc'
Reusable code
Example: Networking
YARP Device Interface
Network proxy
Image Interface
Image Interface
Super-WebCam
Vendor-Supplied
Library/Code
Reusable code
Family of Image Sources
Picolo
framegrabber
DragonFly
fireware camera
“Get an image”
Interface
(IFrameGrabberImage)
OpenCV Grabber
library interface
FFMPEG Grabber
library interface
Server/Remote
network wrapper
TestGrabber
fake images
specific
hardware
widely
supported
libraries
for accessing
image sources
any image source,
on another machine
fake source for
testing
Family of Audio Sources
Microphone
(windows version)
Microphone
(linux version)
“Get a sound”
Interface
(IAudioGrabberSound)
PortAudio
library interface
FFMPEG Grabber
library interface
Server/Remote
network wrapper
specific
hardware
widely
supported
libraries
for accessing
audio sources
any audio source,
on another machine
Family of Motor Controllers
ESD
(Linux,Windows)
Jrkerr
(Windows)
“Control Position”
Interface
(IpositionControl)
specific
hardware
Dimax
(Windows)
Server/Remote
network wrapper
TestMotor
any control interface,
on another machine
fake motors for
testing
Why is this useful?
• Allows collaboration between groups whose
robots have different devices
• Makes device changes less painful
• Devices and communications are orthogonal
features
– Can switch from remote use of device to local use
and vice versa without pain
– Local use can be very efficient, just an extra virtual
method call
iCub Software Interface
• Challenge: a large community of users
• Simple interface
…
IPositionControl *pos;
IEncoders *enc;
IPidControl *pid;
IAmplifierControl *amp;
…
amp->enableAmp(0);
pid->enablePid(0);
pos->positionMove(0,45);
…
pos->positionMove(0,0);
…
amp->disableAmp(0);
pid->disablePid(0);
…
BufferedPort<ImageOf<PixelRgb> >
imagePort;
…
ImageOf<PixelRgb> *image =
imagePort.read();
…
Software Interface
• Access to the hardware (the robot) is made
through software interfaces (a pointer to classes).
The interface is the “contract” between the user
and the system (in theory these won’t change
much – i.e. keep the contact stable).
• Recently improved to keep up with new hardware
and software development:
– ICartesianControl: allows controlling the robot in task
space (position and orientation)
– IAnalogSensor: streams out force feedback and new
finger analog sensors
The iCub code
• Organized as a set of executables glued with
YARP
• Emphasis on modularity with fine granularity
• Modularity and reuse is achieved at three levels:
– C++ libraries: less common and mostly within groups
– “iCub modules”: very often, across groups
– applications: getting more mature, also shared
iCub Modules
•
•
•
The software architecture is made of modules: a module is an
executable that performs a specific task, and whose interface is
defined in terms of YARP ports
Granularity at this level is quite fine
Behaviors are obtained by instantiating and connecting networks of
modules
Modules and Applications
• An application is a logical collection of modules with a particular
purpose (e.g. attention system, reaching, grasping, learning of
affordances, crawling, interaction history, …)
• In practice it is just a collection of configuration files and a text
description in xml
Remotization of applications
A standard to describe applications using xml
<application>
<name> myApplication <\name>
<module>
<name> module1 <\name>
<parameters> … <\parameters>
<node> pc1 <\node>
<\module>
…
<connection>
<to> /input <\to>
<from> /source <\from>
<\connection>
…
Remotization of applications
• Through YARP the application manager performs common tasks
e.g. starting modules with correct parameters, connect and
disconnect ports, check dependencies
• OS independent remote execution, with i/o redirection
.xml
…
…
Software community
•
•
•
•
Several iCub have been delivered
Simulator
Training activities & summer schools
Several laboratories are using the iCub software and/or
YARP
source: www.ohloh.net
…one of the largest
open source teams
in the world, it ranks
in the top 2% of all
projects teams
registered on Ohloh
[email protected]
The entry point to getting help!
Version 2.0 sneak preview
Force/torque measurements
New mechanics
Compliant actuators
The skin
Principle
Lot of sensing points
Structure of the skin
Fingertips
• Capacitive pressure sensor with 12 sensitive
zones
• 14.5 mm long and 13 mm wide, sized for iCub
• Embedded electronics: twelve 16 bit
measurements of capacitance
– either all 12 taxels independently at 50 Hz or an
average of the 12 taxels at about 500 Hz
• A controlled probe (robot arm) moves along a line on
the top of the fingertip
• Measure spatial sensitivity
The measurements of taxel 1,2,3 for different positions
100
Taxel 3
Taxel 2
Taxel 1
90
Sensor measurments(arbitrary units)
80
70
60
50
40
30
20
10
0
2
4
6
Position(mm)
8
10
12
Shoulder joint sensorization
new torque
sensors
hollow wire
exit shaft
FEM analysis

Optimal strain patterns
and sensors placement
have been calculated
with FEM analysis
Model of the elbow sensor/joint
Bode Diagram
-60
-70
-80
-90
-100
-110
-120
-130
-140
-150
-160
90
analitic
experiment
stiff HD
stiff tendon
0
-90
-180
-270
0
10
1
2
10
10
Frequency (Hz)
In: motor voltage; out: measured torque
3
10
Sensor gauging and wiring

Before

After (gauges glued,
10h curing, pads gluing
& wiring)
Equivalent VM strain
Motor
Tensioner
Link
T
T
Strain gauges
Equivalent stress
With Peter Ford-Dominey (INSERM, Lyon)
With a lot of students
@ RobotCub summer school 2008
With Auke Ijspeert, Ludovic Righetti,
Sarah Degallier (EPFL)
With VisLab (IST Lisbon)
With Auke Ijspeert, Ludovic Righetti,
Sarah Degallier (EPFL)
IJCAI – Pasadena, CA
2009 manipulation challenge
With VisLab (IST Lisbon)
RobotCub Summer School 2009
(Alex Maldonado, Federico Ruiz – TUM)
With IST Lisbon, Univ. of Zurich and Genoa
Conclusion
Inspired by this one…
…got this other one!
Giulio Sandini, IIT & U. Genoa
Giorgio Metta, IIT & U. Genoa
Lorenzo Natale, IIT & U. Genoa
Francesco Nori, IIT & U. Genoa
Paul Fitzpatrick, IIT & U. Genoa
Alessandro Scalzo, IIT
Marco Maggiali, IIT
Marco Randazzo IIT
Roberto Puddu, IIT
Gabriele Tabbita, Telerobot
Walter Fancellu, Telerobot
Bruno Bonino, IIT
Fabrizio Larosa, IIT
Claudio Lorini, IIT
Davide Dellepiane, IIT
Mattia Salvi, IIT
Luca Rivano, IIT
Ravinder Dahiya, IIT
Alberto Parmiggiani, IIT
Matteo Fumagalli, IIT
Alexander Schmitz, IIT
Diego Torazza, IIT
Nikos Tsagarakis, IIT & U. Sheffield
Darwin Caldwell, IIT & U. Sheffield
Francesco Becchi, Telerobot
Paolo Pino, Telerobot
Giulio Maggiolo, Telerobot
Gabriele Careddu TeleRobot
John Gray, U. Sheffield
David Vernon, U. Genoa
Rolf Pfeifer, U. Zurich
Harold Martinez, U. Zurich
Gabriel Gomez, U. Zurich
Alexandre Schmitz, U. Zurich
Jonas Ruesch, U. Zurich
Kerstin Dautenhahn, U. Hertfordshire
Chrystopher L. Nehaniv, U. Hertfordshire
Hatice Kose-Bagci, U. Hertfordshire
Frank Broz, U. Hertfordshire
Naeem Assif Mirza, U. Hertfordshire
Lars Olsson, U. Hertfordshire
Qiming Shen, U. Hertfordshire
Cecilia Laschi, SSSA
Paolo Dario, SSSA
Fernando Gamarra, SSSA
Davide Zambrano, SSSA
Egidio Falotico, SSSA
Maria Chiara Carrozza, SSSA
Giovanni Stellin, SSSA
Aude Billard, EPFL
Auke Ijspeert, EPFL
Sarah Degallier, EPFL
Ludovic Righetti, EPFL
Sebastian Gay, EPFL
José Santos-Victor, IST
Alexandre Bernardino, IST
Ricardo Beira, IST
Bruno Damas, ST
Jonas Hornstein, , IST
Luís Vargas, IST
Ricardo Nunes, IST
Hugo Alves, IST
Nuno Conraria, IST
Julio Gomes, IST
Matteo Tajana, IST
Giovanni Saponnaro, IST
Christian Wressengger, IST
Dario Figueira, IST
Rodrugo Ventura, IST
Miguel Praça, IST
Jonas Ruesch, IST
Luís Montesano, IST
Manuel Lopes, IST
Luciano Fadiga, U. Ferrara
Laila Craighero, U. Ferrara
Andrey Olyniyck, U. Ferrara
Livio Finos, U. Ferrara
Giovanni Ottoboni, U. Ferrara
Claes von Hofsten, U. Uppsala
Kerstin Rosander, U. Uppsala
Olga Kochukova, U. Uppsala
Helena Gronqvist, U. Uppsala