Andreas Birk - 3D mapping in marine environments

Transcription

3D Mapping
in Marine Environments
Andreas Birk
Jacobs University
http://robotics.jacobs-university.de
Jacobs University, Robotics Group
http://robotics.jacobs-university.de/
1
Jacobs University
Robotics Group
2
Jacobs Robotics
especially
• 3D Perception (e.g., obstacle avoidance, object recognition)
• 3D Worldmodels (e.g., maps – plus adding semantics)
in unstructured environments
• e.g., marine applications, safety/security/rescue, logistics, …
3
3D Mapping
3D Perception & Semantic Maps
4
Jacobs Robotics
• increasingly active in marine robotics
- due to development of methods that can cope with
- noisy, unreliable sensor data (noise + outliers)
- in unstructured scenes (clutter, occlusions, limited a priori
knowledge, etc.)
• running marine robotics projects
-
EU FP7 MORPH
EU FP7 CADDY
EU H2020 DexROV
two national projects
5
EU FP7:Marine robotics system of self-organizing
logically linked physical nodes (MORPH)
Key MORPH concept:
• self-reconfiguring robot
• consisting of several
closely coupled
vehicles
• for operations in
complex 3D marine
environments
Partners
• ATLAS Elektronik
• Consiglio Nazionale delle Ricerche
• IFREMER
• Jacobs University
• Instituto Superior Tecnico
• Ilmenau University of Technology
• CMRE
• Universitat de Girona
• Institute of Marine Research, IMAR
Jacobs
• online 3D mapping
• view-planning
6
EU FP7: Cognitive autonomous diving buddy (CADDY)
Robot as Divebuddy & Assistent
• Human-Machine-Interaction under
challenging conditions
• applications (endusers): Archeology,
Search & Recovery
Partners
• University of Zagreb
• Consiglio Nazionale delle Ricerche
• Instituto Superior Tecnico
• University of Vienna
• Newcastle University
• Diver Alert Network Europe
Jacobs
• hand/diver detection & segmentation
• mapping as service task
7
EU H2020: Effective Dexterous ROV Operations in
Presence of Communications Latencies (DexROV)
Intelligent Supportfunctions for Teleoperation
e.g., in Oil- & Gasproduction (1.5 – 2.5km depth)
Jacobs Robotics
• recognize & track objects in 3D
• semantic 3D mapping
Partners
• Space Applications Services
• Comex
• UNIGE-ISME
• Idiap Research Institute
• Graal Tech
• EJR-Quartz
8
3D Mapping in Marine Environments
9
Background:
Simultaneous Localization and Mapping (SLAM)
chicken & egg problem
• build map, which needs localization
• while using map for localization
main idea
• roughly localize by
- vehicle motion estimation with odometry / navigation sensors
- plus local sensor data association, aka registration
• optimize localizations/map after registrations to previously
visited places (loop closing) to minimize the cumulative error
10
Overview on the rest of this talk
4 Main Parts
• intro: proper meaning of 3D
• marine "3D" sensors
• 6-DOF registration (aka SLAM front-end)
• short notes on map optimization (aka SLAM back-end)
11
Challenges for 3D Mapping
in Marine Applications
12
3D Map???
No!!!
13
3D: Very Challenging & Important
Bathymetrie is not 3D!!!
• but 2.5D
• i.e., 2D manifold in 3D space
• can not represent 3D objects /
scenarios
e.g.:
• cliffs (biomonitoring)
• oil- & gasindustry
• offshore windparks
• shipwrecks
• harbors
© NOCS February
2007
e.g., MORPH : true 3D
operations by AUVs
getting 3D data is hardly possible from the
surface: you need to explore in 3D!!!
14
Challenges in 3D Underwater Mapping
• real 6 degrees of freedom (DOF)
• localization limited (and costly)
• limitations in range sensors
this all makes
registration
challenging
6 DOF
not really 6 DOF
3D data but motion on a plane (3 DoF)
EU-project MORPH
15
Marine "3D" Sensors
16
“3D” Sensors
• all "3D" sensors deliver 2.5D range
- can be represented as range image, i.e., array of ranges
- this can be exploited for processing
- real 3D only through registration of multiple scans!!!
• different native data formats for scans
- often even directly in 3D, e.g., point cloud
- projecting back to 2.5D then requires sensor model
17
Marine 3D Sensors
• actuated 2D Multibeam Echo Sounder
Blueview BV5000
• solid state 3D Multibeam
Tritech Eclipse
• structured light
• stereo vision
• actuated Laser Range Finder
3D-at-Depth
SL-1
18
Marine 3D Sensors
• actuated 2D Multibeam Echo Sounder
Blueview BV5000
• solid state 3D Multibeam
Tritech Eclipse
• structured light
• stereo vision
• actuated Laser Range Finder
3D-at-Depth
SL-1
19
Underwater Stereo Vision
• underwater & stereo: old technique
- Jean de Wouters d'Oplinter, 1948
stereo tests at Cote D’Azur
- Dimitri Rebikoff, 1954
stereo for mapping archeological sites
payload on manned vehicle Pegasus
• also small own development
-
standard algorithms for dense stereo
(including CUDA version on GPU)
in real time in self-contained system
research contribution: fast registration
Max Pfingsthorn, Heiko Bülow, Igor Sokolovski, Andreas Birk. Underwater Stereo Data
Acquisition and 3D Registration with a Spectral Method. IEEE Oceans, Bergen,
Norway, 2013
input:
2D images
output: 3D
colored point cloud
20
Calibration & Rectification with Pinax Model
• good calibration essential for good stereo data
• in general, current state of the art for calibration is flawed
• flat pane interface
- not a pinhole camera anymore
- but axial camera
• water refraction index
- usually ignored
- but significant influence of
salinity
- if data recorded at different
place than calibration, severe
errors possible
from: Treibitz, T.; Schechner, Y.Y.; Kunz, C.; Singh, H., Flat Refractive Geometry.
Pattern Analysis and Machine Intelligence, IEEE Transactions on , vol.34, no.1,
pp.51,65, Jan. 2012
21
Calibration & Rectification with Pinax Model
• under realistic design constraints (camera front close to flat pane)
• combination of Pinhole with Axial model (PinAx)
- using Axial 12th degree polynomial projection function
- to generate virtual pinhole correction via look-up table (very fast!!!)
- water refraction index (salinity) as parameter (estimated or from CTD)
• calibration simply once in-air
Tomasz Luczynski, Max Pfingsthorn,
Andreas Birk. The Pinax-Model for
Accurate and Efficient Refraction
Correction of Underwater Cameras
in Flat-Pane Housings. (under
review)
22
3D Sonar: Tritech Eclipse
sensing parameters:
•
•
•
•
•
•
•
•
•
Operating Frequency: 240 kHz
Beam Width: 120 deg
Number of Beams: 256
Acoustic Angular Resolution: 1.5 deg
Effective Angular Resolution: 0.5 deg
Depth/Range Resolution: 2.5 cm
Maximum Range: 120 m
Minimum Focus Distance: 0.4 m
Scan Rate: 140 Hz at 5 m, 7 Hz at 100 m
physical parameters:
•
•
•
•
•
•
•
Width: 342 mm
Height: 361 mm
Depth: 115 mm
Weight Wet / Dry: 9 kg /19 kg
Depth Rating: 2500 m
Power Consumption: 60 W
Supply Voltage Nominal: 20-28 VDC
pictures
courtesy
of Tritech, UK
23
Lesumer Sperrwerk
17 scans, ca. 110m x 70m
scan 1
(start angle)
scan 4
scan 17
(end angle)
Google earth
scan 17
24
3D without Range Sensors
there are techniques for full 3D from just 2D camera data (video)
• Structure from Motion (SfM) / Multi-View Registration / Bundle Adjustment
• sometimes aka photogrammetry
disadvantages
• computational complexity (offline)
• non-metric
(just one) example:
T. Nicosevici, N. Gracias, S. Negahdaripour,
and R. Garcia, "Efficient three-dimensional
scene modeling and mosaicing," Journal of
Field Robotics, vol. 26, pp. 759-788, 2009.
25
Registration
26
Registration
problem:
• given two sensor data sets
• find parameters of transform to spatially align them,
• i.e., 3-DoF (2D) or 6-DoF (3D)
• (including uncertainty estimates for SLAM)
27
27
Standard for 3D Registration:
Iterative Closest Point (ICP)
Assume correspondences
by nearest neighbors
• kd-tree for efficiency
Given correspondences
• Horn's algorithm
• for closed form least
squares fit
• using quaternions to get
the rotation part
iterate until convergence
Besl, P. J., & McKay, N. D. (1992). A method for registration
of 3-D shapes. IEEE Trans. on Pattern Analysis and Machine
Intelligence (Vol. 14, pp. 239-256).
Zhang, Z. (1994). Iterative point matching for registration
of free-form curves and surfaces. Int. J. Comput. Vision,
13(2), 119-152. doi: 10.1007/bf01427149.
Horn, B. K. P. (1987). Closed-form solution of absolute
orientation using unit quaternions. Journal of the Optical
Society of America, 4(4), 629-642.
many variants possible, e.g. point to plane
Rusinkiewicz, S., & Levoy, M. (2001). Efficient
variants of the ICP algorithm. Paper presented at
the 3-D Digital Imaging and Modeling. Proceedings.
Third International Conference on.
nice "full package" (i.e., incl. SLAM back end
and visualization)
Nuechter, A., 3D Robotic Mapping. Springer Tracts
in Advanced Robotics (STAR). Springer. 2009
28
Limits of ICP & co
• needs good initial starting conditions
- excellent navigation needed
- hard, respectively costly in underwater applications
• is challenged by partial overlap and noise
example of failed ICP registrations
• Lesumer Sperrwerk dataset
• with Tritech Eclipse data
Heiko Bülow and Andreas Birk. Spectral Registration of
Noisy Sonar Data for Underwater 3D Mapping.
Autonomous Robots, 30 (3), pp. 307-331,Springer, 2011
29
There is more than ICP & co…
• nice and good starting points
- especially, easily understandable for CS-people ☺
- very efficient, highly tuned implementations exist
• but
- they rely on very local information
hence very sensitive to only partial overlap
and disturbances, especially dynamics
- and are iterative methods that require good starting conditions
=> use overall (global) appearance of scans instead
30
"Global" Appearance: e.g. Dominant Planes
3D Plane SLAM
1. consecutive acquisition of 3D range scans
2. extraction of planes including uncertainties
3. registration of scans based on plane sets
•
determine correspondence set
•
•
find the optimal decoupled
•
•
4.
5.
maximizing the global rigid body motion constraint
rotations (Wahba's problem) and
translations (closed form least squares)
embedding in a pose graph
loop detection & relaxation (SLAM proper)
Kaustubh Pathak, Narunas Vaskevicius and
Andreas Birk. Uncertainty Analysis for Optimum
Plane Extraction from Noisy 3D Range-Sensor
Point-Clouds. Intelligent Service Robotics, Vol.3,
Iss.1, p.37-48, Springer, 2010
Kaustubh Pathak, Andreas Birk, Narunas
Vaskevicius, and Jann Poppinga, Fast
Registration Based on Noisy Planes with
Unknown Correspondences for 3D Mapping,
IEEE Transactions on Robotics, 26 (3), pp. 424 –
441, 2010
K. Pathak, A. Birk, N. Vaskevicius, M. Pfingsthorn,
S. Schwertfeger, and J. Poppinga. Online 3D
SLAM by Registration of Large Planar Surface
Segments and Closed Form Pose-Graph
Relaxation. Journal of Field Robotics, Spec.Iss. on
3D Mapping, 2010
31
Planes reasonable in Unstructured Environments
European Space Agency (ESA)
Lunar Robotics Challenge
Teide Volcanic Crater
Tenerife, Spain, 2008
Narunas Vaskevicius, Andreas Birk, Kaustubh Pathak, and Soeren Schwertfeger. Efficient Representation in 3D
Environment Modeling for Planetary Robotic Exploration. Advanced Robotics, Vol. 24, Iss. 8-9, pp. 1169-1197,
Brill, 2010
32
Plane based 6 DoF registration by MUMC
infinite plane:
normal
dist. to origin
plus uncertainty: covaricance
• “left” and “right” view
• two plane sets
• find best correspondences
-
to minimize uncertainty
under rigid body motion constraints
• without combinatorial explosion
-
start with pairs & optimize rot./trans.
max. consensus set with min. uncert.
Minimum Uncertainty
Maximum Consensus (MUMC)
Kaustubh Pathak, Andreas Birk, Narunas Vaskevicius,
and Jann Poppinga, Fast Registration Based on
Noisy Planes with Unknown Correspondences for
3D Mapping. IEEE Transactions on Robotics, 26 (3),
pp. 424 – 441, 2010
33
Plane-Registration on Sonar Data
Lesumer Sperrwerk Dataset
• good correspondence with ground truth
• fringe benefit: high compression
K. Pathak, A. Birk, and N. Vaskevicius. Plane-Based Registration of Sonar Data for Underwater 3D Mapping.
International Conference on Intelligent Robots and Systems (IROS), Taipeh, Taiwan, 2010, pp. 4880 - 4885.
34
Spectral Registration with Multilayer Resampling (SRMR)
• Fast Fourier Transform (FFT) on discretized 3D range data
• decouple translation and rotation and
• resample for spectral registrations in 1D, 2D, 3D to get all 6 DOF
main aspects:
• use of use of phase only matched filters (POMF)
• process whole stack of spherical layers in one step
advantages
• fast fixed computation time
• very robust against noise
• works with partial overlap and significant occlusions
35
sketch of SRMR algorithm
• yaw determination:
-
•
roll-pitch determination:
-
•
resample hemispheres (projection in spherical coordinates) on different radii from the magnitude of the 3D
spectrums
determine the yaw angle by a rotational registration (polar resampled) from the resampled structures (3D
POMF)
re-rotate the 3D spectrum according to the determined yaw angle in order to align the spectrums for yaw
resample hemispheres (rectangular projection) on different radii from the magnitude of the 3D spectrums
determine roll and pitch angle by translational registration from the resampled structures (3D POMF)
translational registration:
-
re-rotate scan data according to all determined angles in order to align the scans for the remaining 3D
translation
determine the 3D translation between the rotationally aligned scans by a 3D POMF registration
H. Bülow and A. Birk. Spectral 6-DOF Registration of Noisy 3D Range Data with Partial
Overlap. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), vol. 35, pp.
954-969, 2013.
36
example results
• underwater sonar: Lesum river lock
• Stanford bunny
• Bremen downtown dataset
• SSRR collapsed car park
37
Collapsed Car-Park dataset
collected at
• 2008 Response Robot Evaluation Exercise
• in Disaster City, College Station, Texas
main sensor:
actuated Laser Range Finder
• SICK S 300
• plus cheap toy servo
38
Collapsed Car-Park: Very Hard for 3D Mapping
• large motions of the robot, i.e., small overlap between scans
• with real 6 DoF, i.e., simultaneous change of roll, pitch, yaw
• without usable motion estimates
- odometry completely off
- gyro severely affected by vibrations
39
Performance of ICP
(not only tested by ourselves but also by experts ☺ )
red dashed lines
indicated
multiple locations
of the facade
in ICP-„registered“
scan pairs
ICP on Crashed Car Parking set: failure for 14 out of 25 scan pairs
40
Comparison of Reg-Methods on Crashed Car-Park
run times
comparison results:
1. SRMR
2. SOFT spectral
3. plane registration
4. ICP
5. principle axis
6. HSM3D
success rates
41
SRMR on Sonar Data
• again: very robust
• and fast (with fixed computation time)
scan 1
(start angle)
scan 17
(end angle)
Heiko Bülow and Andreas Birk. Spectral Registration of Noisy Sonar Data for Underwater 3D
Mapping. Autonomous Robots, 30 (3), pp. 307-331,Springer, 2011
42
Back-end: Loop Closing and Optimization
43
main idea
• roughly localize by navigation and registration
• optimize localizations/map by registration with previously visited
places (loop closing) and minimize the cumulative error
now (very short) glimpse on 2nd aspect
44
Loop Closing
• proximity based
- easiest possible strategy: use current localization estimate
- to check whether there are previously visited places around
- if so: try registrations
• place recognition
- non-trivial, especially with respect to good strategies to get
reasonable cost/benefit
- typically (visual or 3D) feature collections in associative
representation (hashes)
45
Place Recognition
• e.g., FAB-MAP
- monocular images
- based on bag of words on SURF
- very efficient (linear in #places)
Cummins, M., & Newman, P. FAB-MAP: Probabilistic Localization and
Mapping in the Space of Appearance. The International Journal of Robotics
Research, 27(6), pp. 647-665. 2008
Cummins, M., & Newman, P. Appearance-only SLAM at large scale with
FAB-MAP 2.0. The International Journal of Robotics Research, 30(9), pp. 11001123. 2011
• extension to stereo
- both 2D visual features (SURF)
- plus shape features
Ivaylo Enchev, Max Pfingsthorn, Tomasz Luczynski, Igor Sokolovski, Andreas
Birk, Daniel Tietjen. Underwater Place Recognition in Noisy Stereo Data
using Fab-Map with a Multimodal Vocabulary from 2D Texture and 3D
Surface Descriptors. IEEE Oceans. Genoa, Italy, 2015
46
SLAM: optimization part
use loop closures as "feedback" to minimize error
• "historically": Kalman Filter SLAM
- n features => n2 variables
- zero-mean white Gaussian noise assumed
- data association (how to match features)
• hence alternatives popular
- e.g., particle filter aka “condensation”, “(sequential) Monte
Carlo” or “survival of the fittest“ algorithm
- e.g., graph based
47
Uncertainty/Confidence Estimates
in Registrations for SLAM
very useful as "weights" for each registration result
to measure the "quality" of the spatial estimates
• well established for ICP & co
• inherent in plane-registration
• but not studied for 3D spectral methods
=> extension for SRMR registration
Max Pfingsthorn, Andreas Birk, and Heiko Buelow, Uncertainty Estimation for a 6-DoF Spectral
Registration method as basis for Sonar-based Underwater 3D SLAM, International
Conference on Robotics and Automation (ICRA), Saint Paul, Minnesota, IEEE Press, 2012
48
Generalized Graph SLAM
handling Ambiguities
• Local Ambiguity:
- Ambiguity in sequential observations
- Spatially and temporally close
• Global Ambiguity:
- Ambiguity in non-sequential observations, i.e. loops
- Spatially close, temporally far
Max Pfingsthorn and Andreas Birk. Simultaneous Localization and Mapping (SLAM) with Multimodal Probability Distributions.
International Journal of Robotics Research, 32(2), pp. 143-171, Sage, 2013
Max Pfingsthorn and Andreas Birk. Generalized Graph SLAM: Solving Local and Global Ambiguities through Multimodal and
Hyperedge Constraints. International Journal of Robotics Research (IJRR). Sage, 2015
49
Local Ambiguity
or can arise from complementary motion estimates:
one fails => mutually exclusive choices better option than probabilistic fusion
50
Global Ambiguity example: Loop Detection
place recognition, here with FabMAP, can give multiple results
=> again: need for representing alternative motion options
51
Representation as Multimodal Hypergraphs
mid-stage:
SLAM
front
end
multimodal
hypergraph
Generalized
Prefilter
(discrete
optimization)
standard
pose-graph
SLAM
back
end
• new discrete optimization stage between front- and back-end
• generate standard graph without ambiguity
find most globally consistent component combination
by traversing a spanning tree
with heuristics to fight curse of combinatorial explosion
52
Generalized Graph SLAM
multiple alternative spatial relations in one probability density:
• local ambiguity as Mixture of Gaussians (MoG)
• global ambiguity as mixture of constraints in a hyperedge
from registration
from loop detection
hyperedge weights
one MoG per hypothesis in hyperedge
53
Results of Generalized Graph SLAM
vs state of the art methods
Sünderhauf’s
Switchable
Constraints
Agarwal’s
Dynamic
Covariance
Scaling (DCS)
Olson’s MaxMixture
Latif’s Realizing, Reversing,
Recovering (RRR)
Generalized Graph SLAM with Prefilter
54
Short Final Note
also work on how to
• plan to efficiently generate the maps (exploration)
• ensure good coverage (view-planning)
• make sense of them (object/terrain classification & semantic mapping)
55
Conclusions
• bathymetry is not 3D!!!
• 6 DoF registration
-
complementing (or even replacing) navigation
there is more than ICP
there are alternatives that are much more robust (and faster)
e.g., plane based registration with MUMC
e.g., spectral registration with SRMR
• SLAM back-end
- Generalized Graph SLAM
- can handle outliers in local motion estimates and loop closures
- e.g., short loss of navigation or ambiguous place recognition
• efficient map generation & making sense of them
56

Andreas Birk - 3D mapping in marine environments

Transcription

Similar documents

Invitation - Demcon 15-10-2015 - LEO Center for Service Robotics

Program RoboCafé LEO Robotics | Demcon 15-10-2015

pdf presentation

DEMCON develops robot for automatic asparagus harvesting

mBot Edison eBot

summit europe - Singularity University

Prior Lake Robotics Middle School Program

The Voice: Brittany Kennell - Lester B. Pearson School Board

ComQuest Reports - ComQuest Academy Charter High School

4.0 first robotics - Team 4946, ALPHA DOGS