Stroboscopic Image Synthesis of Sports Player from Hand

Comments

Transcription

Stroboscopic Image Synthesis of Sports Player from Hand
Stroboscopic Image Synthesis
of Sports Player from
Hand-Held Camera Sequence
Kunihiro Hasegawa, Hideo Saito
(Keio University)
Overview
Synthesis of a stroboscopic image for sports analysis
using hand-held camera
2
Agenda
1.
Background of our study
2.
Proposed method
3.
Experiments and results
4.
Conclusion
3
Background

A variety of sport analysis studies using image
have been performed
 Soccer
(Hamid)[1]
 Basketball (Lu)[2]
 American football (Atmosukarto)[3]
Hamid et al.
Lu et al.
Atmosukarto et al.
[1] R. Hamid et al., “Player localization using multiple static cameras for sports visualization” CVPR 2010.
[2] W.-L. Lu et al., “Learning to Track and Identify Players from Broadcast Sports Video” PAMI 2013.
[3] I. Atmosukarto et al.,“Automatic Recognition of Offensive Team Formation in American Football Plays” CVPRW 2013
4
Background

Our target : athletic running
 Main

target : Measurement of speed and stride length
Previous work : Synthesis of a stroboscopic image
to measure them [4]
Result of previous work
 Main

Problem : High computational cost
Solution : Change a method of synthesis image
[4] K. Hasegawa et al., “Auto-Generation of Runner’s Stroboscopic Image and Measuring Landing Points Using a Handheld Camera,” IMVIP 2014.
5
Background

Purpose
 Proposition
of a method for synthesizing the stroboscopic
image with computationally efficient algorithm

Examples of use case
 Measurement
of a runner’s speed and stride length
(main use came)
 Analysis of a player’s motion

Problem
 How
we reduce an amount of computation?
6
Method - Abstract
Input images
Synthesize
a background image
Overlay frame image
on the background image
Create a player’s
mask image
Remove the player’s shadow
by Mean-Shift
Last frame?
Overlay the player on
the background image using the mask
No
Yes
Go to next frame
Stroboscopic image

Proposed method has two steps.
 Synthesis
of a background image (red)
 Synthesis of stroboscopic image (green)
 Removal of player’s shadow (black)
7
Environment of capturing video



A person controlling a camera (= operator) is still
The operator stands a few tens of meters away from a
center of a track
A Background is static
8
Synthesis of Background

How we reduce an amount of computation?
 Previous
method : using Mean-shift to each pixel
-> Cause of the high computational cost
 Solution : Stitching images with masking the bounding
box of the player
-> Computational cost of stitching is low.
Original image
Masked image
9
Synthesis of Background

How we make the mask and synthesize?
 SURF
is used as feature points for homography
 HOG-SVM detects the bounding box of the player
Input video sequence
Background image
10
Synthesis of stroboscopic image

Utilizing a mask of the player’s shape for overlaying
 Previous
method : blending with the player’s area
-> Pixel value become almost 0 in the worst case
 Proposed method :


The Mask image has the only player
Obtaining by subtraction between the background image
and the input frame image
Example of a mask for overlaying a player
11
Synthesis of stroboscopic image

How we synthesize the stroboscopic image with the mask?
1. Overlaying the input frame image to the background image
Input frame image
Overlaid image
Background image
12
Synthesis of stroboscopic image

How we synthesize the stroboscopic image with the mask?
2. Making a player mask image using overlaid image and background image.
Overlaid image
Subtraction
Binarization
Runner mask image
Background image
13
Synthesis of stroboscopic image

How we synthesize the stroboscopic image with the mask?
3. Overlaying player using the mask.
Runner mask image
Overlaying × N(frames)
Stroboscopic image
Input frame image
14
Removal of player’s shadow

Connecting shadows to the players may occur
negatively affects
 Using
a synthesis method utilizing Mean-Shift
 We select this removal area manually
Stroboscopic image with shadow
15
Experiments and results

We performed four experiments
 Comparison
with our previous method
 Visualizing and accuracy
 Comparison with other methods
 Application to other scenes

Conditions
 Image
size : 640 * 480 pixels
 CPU:Intel CORE i 7 2.40GHz / 4 cores
 Memory:8.00GB
16
Experiments and results

Comparison with our previous method
Processing time of synthesizing a background image with three frames
Average
Maximum
Minimum
Previous method
Previous(sec.)
2.44×103
2.53×103
2.35×103
Proposed(sec.)
1.72×101
1.84×101
1.68×101
Proposed method
17
Experiments and results

Comparison with other methods
Proposed method

Autostitch
Frame subtraction
Visualization and accuracy
Visualization of a runner’s footprints
 Error
of measurement : 0.13m (average)
18
Experiments and results

Application to other scenes
Walking
Downhill skiing
Figure skating
Speed skating
19
Experiments and results

Application to other scenes
Walking
Downhill skiing
Figure skating
Speed skating
20
Conclusion

Proposition of a method of synthesizing a
stroboscopic image

Succession of speed up the processing of background
image synthesis

More improvement needs for full automation.
21

Similar documents