AR Visualization of Wheel Trajectories of Driving Vehicle by Seeing

AR Visualization of Wheel Trajectories of Driving Vehicle by
Seeing Through Dashboard
Shota Sasai Itaru Kitahara† Yoshinari Kameda† Yuichi Ohta†
†Graduate School of System and Information Engineering University of Tsukuba
E-mail: [email protected], † {kitahara,kameda,ohta}@image.iit.tsukuba.ac.jp
This paper proposes a method to avoid car accidents such as running off the lane or hitting objects by
projection-based AR technique that visualizes blind regions in front of a driving vehicle. As the result,
smooth passing each other even on a narrow road might be realized.
One of the general methods to visualize the blind region around a vehicle is to set cameras at the body
of the vehicle such as side mirrors and display the captured images on the in-vehicle screen such as a
monitor of car-navigation system. However, in this method, it is difficult to understand the spatial
relationship between the displayed images and the surrounding environment, because the position and
orientation of the capturing camera are different from the driver’s viewpoint. Moreover, it is difficult to
perceive the 3D spatial information by observing a monocular image.
In order to visualize the blind region, our method sees-through the dashboard of a vehicle by using
projection-based diminished reality technique, and superimposes the virtual wheel trajectories by using
AR fashion. Figure1 shows the overview of our proposed system. A view ahead of the vehicle is
transformed to the driver’s view by homography transformation. Then, the driver’s view is projected
onto the dashboard by using a projector set on near the driver’s viewpoint. As the result, the view ahead
of the vehicle can be transparently observed. The virtual wheel trajectories are overlaid on the seethrough image. Thanks to AR displaying, the geometric consistency between the presented images and
the surrounding environment is realized so that the driver can easily understand their spatial
relationship. Moreover, the overlaid wheel trajectories make drivers easily understand the 3D spatial
information even by observing a monocular image.
We developed a pilot system of our proposed method, and conduct on subjective evaluations to confirm
the effectiveness of our proposed method.
Projector
Wheel Trajectory
See-Through Image
Dashboard
Camera
Blind Region
Figure1 : AR Visualization of Wheel Trajectories of Driving Vehicle by Seeing Through Dashboard
AR Visualization of
Driving Vehicle’s Wheel Trajectories
by Seeing-Through Dashboard
Graduate School of Systems and Information Engineering
University of Tsukuba
Shota Sasai, Itaru Kitahara, Yoshinari Kameda, Yuichi Ohta
1
Background and Purpose
In order to safely drive …
A lot of information around the vehicle,
More visually understanding,
the better.
Our target:
Blind regions in front of the vehicle
We aim to :
- Prevent car accidents such as
running off the lane or hitting objects
- Understand a width of the driving vehicle
by using AR
2
Conventional Method
•Around View Monitor [1]
- Capturing images as if the driver looks down the vehicle.
- Displaying the images on the in-vehicle screen.
However…
It is difficult to understand the spatial relationship
The displayed images and
The surrounding environment
[1]NISSAN MOTOR CORPORATION 「Around View Monitor」
3
Related Works
・Seeing-through by using projection AR technique
The image is captured by a camera
fixed behind the object.
Transform
Projected to the retro-reflective sheets
Improve the visibility of the projection images
•Apply the See-through technique to the vehicle
[2]
In order to safely drive …
－A lot of information around the vehicle
－More visually understanding
[2]Takumi Yoshida, et al.(2009). “A Display System for Vehicle Blind Spot Information Using Head Tracked Retro-reflective Projection
Technology” The Journal of The Institute of Image Information and Television Engineers vol.63 No.6 P.801-809
[3],Inami M, et al.(2000). “Visuo-haptic display using head-mounted projector” Virtual Reality, 2000. Proceedings. IEEE P.233-240
4
Our Approach
See-through the dashboard by projection-based
diminished reality technique
＋
AR visualization of wheel trajectories
System Overview
Projector
Wheel Trajectory
See-Through Image
Dashboard
Camera
Blind Region
5
Components of the System
The camera
The projected surface
The projector
The accelerometer
The PC
6
Processing Flow
① Geometric correction of projection images for the surface
Correction
②
See-through the dashboard
See-through
③ Calculate and overlay wheel trajectories
Merging
3 processes
7
Geometric Consistency of Projection Images
Projection Image
Homography Transformation ②
Virtual Display
Subregion ②
Subregion ②
Without
correction
correction
Subregion ①
Subregion ①
Subregion ③
Subregion ③
Homography Transformation ①
Retro-reflective sheets on the dashboard Homography Transformation ③
Improve the visibility of the projection images
8
See-through the Dashboard
A view ahead of the vehicle
［Display the image without see-through ］
The view
through a windshield
The image Projected
on the dashboard
After
9
See-through the Dashboard
Homography Transformation
R’’3
R’’4
R’3
R’4
R’’1
R’’2
R’1
R’2
Observer’s viewpoint
Virtual display
on the dashboard
R’’
Camera
The view
through a windshield
The image Projected
on the dashboard
Road surface
of blind rigions
R’
R
10
See-through the Dashboard
A view ahead of the vehicle
［A result of see-through ］
The view
through a windshield
The image Projected
on the dashboard
Homography Transformation
Before
After
11
Calculate the Wheel Trajectories
Calculation by trigonometric function
θ
Similar calculation is applied to the other wheels.
[Assumption]
The wheels do not sideslip,
because this system is used in very slow speed.
θ
12
Overlay the Wheel Trajectories
Acquire the steering angle
by an accelerometer.
Accelerometer
Generate the CG of
the front and rear wheel trajectories.
Overlay the CG on the see-through image.
Green line・・・front wheel trajectories
Red line ・・・rear wheel trajectories
13
System Demonstration
A view from the passenger seat
Green line・・・front wheel trajectories
Red line ・・・rear wheel trajectories
14
Subjective Evaluations
Effectiveness of proposed method
Even on a narrow road
・Understand a width of the driving vehicle
・Prevent car accidents such as
running off the lane or hitting objects
［Experimental Procedure］
・Participants (16 people) observe the image
from a passenger seat.
・An experimentaler drives the car.
15
Experimental Procedure
［Driving Courses］
4.80[m]
4.80[m]
A driving vehicle
1.70 [m]
Obstacle
2.40[m]
Course ①
A straight course with a obstacle
For evaluation
“Understanding of a width of
the driving vehicle”
Trial 1
Course ②
A right angle course
4.12[m]
For evaluation
“Prevention effect of running
off the lane and hitting objects “
Trial 2
Trial 3
(proposed method)
Trial 4
(proposed method)
Wheel trajectories
OFF
ON
OFF
ON
Driving course
Straight
Straight
Right angle
Right angle
Participants answer a quastionnaire each trials
16
Experimental Procedure
［Quastionnaire］
Q.1：Understanding of the driving vehicle width
Q.2：Understanding of the spatial relationship
between wheels and the roadside
Strongly
Agree
Agree
Neutral
Disagree
4
3
2
1
Trial 1
Trial 2
Trial 3
(proposed method)
Strongly
Disagree
0
Trial 4
(proposed method)
Wheel trajectories
OFF
ON
OFF
ON
Driving course
Straight
Straight
Right angle
Right angle
Participants answer a quastionnaire each trials
17
Q.1：Understanding of the driving vehicle width
Trial 1
Trial 2
Trial 3
(proposed method)
Trial 4
(proposed method)
Wheel trajectories
OFF
ON
OFF
ON
Driving course
Straight
Straight
Right angle
Right angle
A result of Question1
Value
4
3
2
1
0
Trial１
Wilcoxon signed-rank test
Trial２
Trial３
Value Number
Trial４
Q.1:There was a significant difference (p<0.01)
both of Straight course with a obstacle and Right angle course
18
Q.2：Understanding of the spatial relationship
between wheels and the roadside
Trial 1
Trial 2
Trial 3
(proposed method)
Trial 4
(proposed method)
Wheel trajectories
OFF
ON
OFF
ON
Driving course
Straight
Straight
Right angle
Right angle
A Result of Question2
4
Value
3
2
1
0
Trial１
Trial２
Trial３
Trial Number
Trial４
Wilcoxon signed-rank test
Q.2:There was a significant difference (p<0.01)
both of Straight course with a obstacle and Right angle course
19
Q.2：Understanding of the spatial relationship
between wheels and the roadside
Trial 1
Trial 2
Trial 3
(proposed method)
Trial 4
(proposed method)
Wheel trajectories
OFF
ON
OFF
ON
Driving course
Straight
Straight
Right angle
Right angle
A Result of Question2
4
Value
3
2
1
0
Trial１
Trial２
Trial３
Trial Number
Trial４
[comment of a participant]
”The virtual wheel trajectories disappeared when the steering was turned a lot.”
［Solution］ Expand the display area
20
Conclusion
• We proposed a method to see-through the dashboard
and overlay the virtual wheel trajectories.
• We confirmed the effectiveness of our method.
Even on a narrow road:
-Understand a width of the driving vehicle.
→ Smooth passing each other.
-Prevent running off the lane.
• It is difficult to prevent hitting objects
when the vehicle curves
because the display area is small.
21
Future Work
• Expansion of the display area.
• Face-Tracking for the driver.
• Subjective evaluations driven by participants.
22