AR_at_VTT_brief_with_+ - VTT Virtual project pages

Augmented Reality
Applications at VTT
Charles Woodward
VTT Technical Research Centre of Finland
Introduction - AR
Augmented Reality (AR)
Related technologies
Adding of virtual objects in view of real world
C.f. Virtual Reality (completely virtual worlds)
Properties: real-time, interactive, 3D registration
<-> Challenges: tracking, accuracy, speed, mobile
Augmented Virtuality
Mediated Reality
Diminished Reality, …
<-> Mixed Reality (MR)
Reality /
Video image
Augmented
Reality
Virtual Reality /
Computer graphics
1
AR at VTT
Augmented Reality / 3D Tracking research at VTT
Established in 2000, team based in Espoo & Oulu
AR/MR core technology & high-end applications
Supporting / application teams around VTT
Leading edge technology
First in the world with various AR applications
Licensing for independent developers (ALVAR SDK,
open source & commercial versions)
See www.vtt.fi/multimedia
Applications, incl.
Games, Entertainment
Print media, Advertising
Industrial operation, Assembly
Interior design, Furniture, Rendering
Architecture, Engineering & Construction (AEC)
Virtual Worlds Collaboration, Telepresence
Early Applications: Games & Entertainment
Vappunaamari.com
SymBall
CamBall
https://youtu.be/7SWEl1Uen3M
TelePet
StereoGames
2
Consumer Applications: Print & Advertising
Dibidogs in Katso &
AR children’s book
Nestle bunny on
cocoa package
https://youtu.be/aR-g9RbYUYs
Sparkly in Elle magazine
https://youtu.be/AjDjsmr0G14
Augmented business
cards (Painonet)
DNV (Det Norske Veritas):
Technology Outlook 2020
Veijarit & Vares movie
trailers in Seiska magazine
https://youtu.be/s1WfsnjJdu4
Interior Design
VividAR
AR interior design system by VividWorks, based on VTT technology
Free download, various furniture retail companies around the world
https://youtu.be/lp5plNOT20U
Incl. Diminished Reality
3
Photorealistic Rendering
Challenges, e.g.
Materials, incl. reflections, transparency
Lighting: direction, intensity, colors, soft
shadows
Accounting for real world lighting conditions
Matching computer graphics with webcam
video quality
Comparisons
Left - standard 3D graphics
Right - using VTT methods
https://youtu.be/fhFzStkoE50
Mixed Reality Telepresence
Connecting Real and Virtual Worlds
Mixed Reality teleconferencing between real
people and Second Life avatars
Avatars appear at meeting room with real
people, following real people’s gestures
Implemented in co-operation with IBM Corp. (USA)
https://youtu.be/DNB0_c-5TSk
4
Industrial Applications: Production Sites
Augmented maintenance, operation, training
Plant asset management: Simantics Mobile
Robot simulations in real environments
Partners: Wärtsilä, Metso, Valtra, Cargotec, Fortum, ..
Augmented Assembly
Project AugAsse
Data Pipeline
Pilot case
3D puzzle
Valtra (tractor company)
Hydraulic block (200 pcs)
3D CAD
STEP
DFA
CG
AR
https://youtu.be/QSwZ5CINaPk
5
AR Maintenance
Project S-STEP
Smart technologies for lifecycle performance
Knowledge sharing solutions for field service personnel
Collecting tacit knowledge, multimodal interaction, mobile AR
Industrial partners: Konecranes, KONE, Wärtsilä, others
VTT tasks, pilots
AR maintenance
Remote assistance
Wearable interfaces
User experience, acceptance
AR Applications in Space
Project EdcAR
Commission for European Space Agency (ESA)
VTT with partners TAS-I, TAS-F, ICCS
Budget 0,5 M€, duration 2015-2017
AR solutions for space domain
maintenance
Specification of AR solutions to address
identified critical tasks and activities
Selection of suitable technology components
Demonstrators for identified use cases
6
Architecture and Construction
AROnSite system
Features: Photorealistic rendering, 4D etc.
Devices:PCs, mobile phones, tablets
Application cases, incl.
Skanska offices in Ruskeasuo, Helsinki
Hotel complex, Billnäs Ironworks at Raseborg
Kämp Tower, Jätkäsaari in Helsinki
First in the world to involve mobile AR at construction sites,
and in real urban decision making cases
https://youtu.be/c1DUkVhqVsM
https://youtu.be/wvSPzG7AqLY
https://youtu.be/rVt86NGXQv4
Some Recent Cases
Applications
Real estate, marketing
Citizen engagement, participatory design
7
Markerless 3D Tracking by VTT
Markerless tracking
Early experiments, Forchem case 2010
Model-based initialization + real time feature tracking
Point cloud based tracking
Properties: automatic, accurate, fast, robust
Single solution to handle wide range of applications
International licensing: Inglobe (Italy) / AR-Media
https://youtu.be/WOVjISxlhpU
https://youtu.be/xpZDXih9zLg
https://youtu.be/EUmNbNa3RYY
Indoor Navigation
Tracking solutions
Point clouds obtained from photos
Using Photoscan / VisualSFM tools
Dynamic loading of point clouds
Updating of point clouds, edge computing
Autonomous robot to collect & update point clouds
Example
Navigation to vending machines, find food items
Operating instructions, showing which buttons to
press
https://youtu.be/ttgHKtyKHKw
8
Facility Asset Management
Project DigiSpaces
Mobile AR visualization and interaction with Building
Information Models (BIM) & service book
Applications: building's life cycle services, e.g.
maintenance & repair
Partners: Skanska, Granlund, Pöyry, Tekla, Solibri, Nokia
Functionality
Navigation to service targets
Interacting with targets by clicking in camera view
“X-ray vision”, seeing targets behind walls etc.
Tracking
Based on 3D point clouds
Supported by IMU, locationing using Nokia’s HAIP
User tests
Collecting user requirements
Evaluation, most positive feedback
https://youtu.be/uYFtYbqvoq0
Mobile AR for Lean Construction
Project VisiLean
In co-operation with Aalto University
Combining lean production practices with
BIM, visualization by AR
Point clouds derived from panoramic images,
coupled with photogrammetry data
Web based, device independent user
interface, AR mode for all functionality
9
World Browsing
Task
Augmenting the camera view with point-of-interest
(POI) information
E.g. tourist info, mobile advertising
Challenge
Current world browser solutions are based on GPS
and compass, ínaccurate, bad user experience
Accurate tracking of mobile device (both location
and rotation) required
Solution
3D point cloud based tracking
Real-time implementation optimized for mobile
phones
Providing extreme accuracy, robustness and speed
Technology licensing to Nokia
Visualizing the Past
Augmenting historical photos
Pilot cases in Turku and Helsinki
Augmenting historical photos at real locations
Showing how city used to look in the good old times
Projects
Futuristic History & Street Smart (EIT Digital)
https://youtu.be/5ZLNjnnXinA
10
ALVAR Subroutine Library
”A Library for Virtual and Augmented Reality”
Computer vision based 3D tracking for AR
World leading performance, accuracy, robustness, speed
ALVAR Desktop
Windows / Linux – marker based – Open Sourced
Users incl. Google, NASA, MIT, Vuzix, Columbia Univ., ROS
https://youtu.be/xc-4B4CljHI
ALVAR Mobile
Marker, multimarker and 2D image tracking
iOS, Android, Windows Phone, Windows, Mac, Flash, Silverlight
https://youtu.be/hWwlxBefg_0
ALVAR Tracker
Point cloud based 3D tracking, SLAM, sensor fusion …
iOS, Android, Windows Phone, Windows, Mac
ALVAR Unity Plugin
Content authoring, point cloud editing tools
https://youtu.be/ttgHKtyKHKw
VTT Tracking in Space, by NASA, Google, …
NASA Spheres robots in zero-gravity
Co-operation with NASA and Google
ALVAR multimarkers used for evaluating ground truth of Project Tango tracking
See Google video https://www.youtube.com/watch?v=Vc5YyLl1Ksg&t=130
See NASA video: https://www.youtube.com/watch?v=psZ6UAFUOT4&t=305
NASA Hedgehog robot in micro-gravity
Co-operation between NASA, Stanford and CalTech
ALVAR multimarkers used for orientation tracking of robot, under fast movement
See news & videos: http://www.dailymail.co.uk/sciencetech/article-3222992/Nasareveals-bizarre-hedgehog-robot-roll-fall-alien-planets.html
11
Contact:
Dr. Charles Woodward, VTT
[email protected]
http://www.vtt.fi/multimedia
www.youtube.com/user/VTTAugmentedReality
TECHNOLOGY FOR BUSINESS
12