Complexity-Adaptive Distance Metric for Object Proposals Generation

Transcription

Complexity-Adaptive Distance Metric for Object Proposals Generation
Yao Xiao
Cewu Lu
Efstratios Tsougenis
Yongyi Lu
Chi-Keung Tang
The Hong Kong University of Science and Technology
{yxiaoab,lucewu,tsougenis,yluaw,cktang}@cse.ust.hk
1. Supplementary Experiments
In this document, all the comparison experiment results in terms of various set up are presented for evaluation.
1.1. Evaluation on PASCAL VOC 2012 Dataset
First we evaluate the performance on PASCAL VOC 2012 dataset. Figure 1 shows the recall versus IoU threshold curves
with candidates number varying from 500-3000. The quantitative results of corresponding MABO and AUC are reported in
Table 1. In Figure 2 the recall versus candidate number curves under IoU threshold ranged from 0.5 to 0.9 are displayed, as
well as the area under ”recall versus IoU threshold” curves for varying number of proposed windows.
To show the robustness and generality of the proposed complexity-adaptive distance measurement, we further compare
the recall values for each of the 20 categories, as visualized in Figure 3. For details of the experiment set up please refer to
Section 4 of the paper and the corresponding references.
Methods
Selective Search [3]
MCG [1]
EdgeBox [4]
SPA [2]
CA1
CA2
500 Candidates
MABO
AUC
0.771
0.517
0.757
0.510
0.755
0.520
0.736
0.487
0.768
0.517
0.775
0.536
1000 Candidates
MABO
AUC
0.799
0.562
0.782
0.547
0.782
0.559
0.776
0.545
0.809
0.585
0.812
0.597
1500 Candidates
MABO
AUC
0.808
0.578
0.795
0.566
0.792
0.576
0.796
0.577
0.828
0.620
0.829
0.627
2000 Candidates
MABO
AUC
0.812
0.585
0.802
0.578
0.798
0.585
0.800
0.583
0.836
0.631
0.840
0.647
2500 Candidates
MABO
AUC
0.840
0.642
0.831
0.634
0.801
0.591
0.800
0.583
0.838
0.634
0.851
0.661
3000 Candidates
MABO
AUC
0.843
0.647
0.837
0.642
0.803
0.595
0.800
0.583
0.843
0.641
0.854
0.665
time
5.4
33.4
0.3
16.7
7.3
15.6
Table 1. Comparison results of MABO and AUC on PASCAL VOC 2012 dataset, with candidates number varying from 500-3000; CA1
and CA2 respectively are the two parameter settings used in our method. Running time of all the methods are also shown.
1.2. Evaluation on BSDS Dataset
We further do comparisons on BSDS dataset, results are presented as follows. Figure 4 shows the recall versus IoU with
candidates number varying from 500-2000. The quantitative results are reported in Table 2. In Figure 5 the recall versus
candidate number curves under IoU threshold ranged from 0.5 to 0.9 are displayed, as well as the area under ”recall versus
IoU threshold” curves for varying number of proposed windows. From this comparisons we can see that our proposed method
also achieves state-of-the art results.
Methods
Selective Search [3]
MCG [1]
EdgeBox [4]
CA1
CA2
500 Candidates
MABO
AUC
0.731
0.512
0.734
0.531
0.733
0.514
0.737
0.524
0.721
0.512
1000 Candidates
MABO
AUC
0.766
0.564
0.769
0.581
0.758
0.542
0.771
0.580
0.766
0.583
1500 Candidates
MABO
AUC
0.781
0.584
0.784
0.597
0.767
0.552
0.786
0.601
0.786
0.610
2000 Candidates
MABO
AUC
0.784
0.586
0.785
0.600
0.772
0.557
0.789
0.605
0.801
0.631
Table 2. Comparison results of MABO and AUC on BSDS dataset, with candidates number varying from 500-2000; CA1 and CA2
respectively are the two parameter settings used in our method. Running time of all the methods are also shown.
1
References
[1] P. Arbeláez, J. Pont-Tuset, J. T. Barron, F. Marques, and J. Malik. Multiscale combinatorial grouping. CVPR, 2014. 1
[2] P. Rantalankila, J. Kannala, and E. Rahtu. Generating object segmentation proposals using global and local search. The IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), June 2014. 1
[3] J. R. Uijlings, K. E. van de Sande, T. Gevers, and A. W. Smeulders. Selective search for object recognition. International journal of
computer vision, 104(2):154–171, 2013. 1
[4] C. L. Zitnick and P. Dollár. Edge boxes: Locating object proposals from edges. In Computer Vision–ECCV 2014, pages 391–405.
Springer, 2014. 1
1
0.8
0.8
0.6
0.6
0.4
0.2
0
0.5
Recall
Recall
1
CA1
CA2
SS
MCG
EB
SPA
0.6
0.4
0.2
0.7
0.8
IoU overlap threshold
0.9
0
0.5
1
1
1
0.8
0.8
0.6
0.6
0.4
0.2
0
0.5
CA1
CA2
SS
MCG
EB
SPA
0.6
0.4
0.2
0.7
0.8
0.9
0
0.5
1
1
1
0.8
0.8
0.6
0.6
0.2
0
0.5
CA1
CA2
SS
MCG
EB
SPA
0.6
0.4
0.2
0.7
0.8
(e) 2500 proposals per image.
0.7
0.8
0.9
1
0.9
1
0.9
1
CA1
CA2
SS
MCG
EB
SPA
0.6
0.7
0.8
(d) 2000 proposals per image.
Recall
Recall
(c) 1500 proposals per image.
0.4
0.6
(b) 1000 proposals per image.
Recall
Recall
(a) 500 proposals per image.
CA1
CA2
SS
MCG
EB
SPA
0.9
1
0
0.5
CA1
CA2
SS
MCG
EB
SPA
0.6
0.7
0.8
(f) 3000 proposals per image.
Figure 1. Recall versus IoU threshold on PASCAL VOC 2012.
1
0.8
0.8
0.6
0.6
CA1
CA2
SS
MCG
EB
SPA
0.4
0.2
0
0
Recall
Recall
1
500
1000
1500
2000
Number of proposals
2500
CA1
CA2
SS
MCG
EB
SPA
0.4
0.2
0
0
3000
500
1
1
0.8
0.8
0.6
0.6
CA1
CA2
SS
MCG
EB
SPA
0.4
0.2
0
0
500
1000
1500
0.6
2000
Number of proposals
2500
0
0
3000
CA1
CA2
SS
MCG
EB
SPA
500
1000
1500
2000
Number of proposals
2500
3000
1
CA1
CA2
SS
MCG
EB
SPA
0.8
0.2
0.6
CA1
CA2
SS
MCG
EB
SPA
0.4
0.2
500
3000
(d) IoU = 0.8.
0.4
0
0
2500
0.2
Area under recall
Recall
0.8
2000
0.4
(c) IoU = 0.7.
1
1500
Number of proposals
(b) IoU = 0.6.
Recall
Recall
(a) IoU = 0.5.
1000
1000
1500
2000
Number of proposals
(e) IoU = 0.9.
2500
3000
0
0
500
1000
1500
2000
Number of proposals
2500
3000
(f) Area under ”recall versus IoU threshold” curves.
Figure 2. (a)-(e) Recall versus candidate number curves on PASCAL VOC 2012, with IoU threshold ranged from 0.5 to 0.9. (f) Area under
”recall versus IoU threshold” curves which are shown in Figure 1, for varying number of proposed windows.
vg
0.4
0.4
0.2
0.2
0
0
0.4
0.2
0.2
0
0
0.8
0.6
CA1
CA2
SS
MCG
EB
SPA
(c) 2000 proposals, IoU = 0.8.
fa
ca
do t
pl g
an
ta e
bl
tr e
ai
ho n
r
m se
bi
ke
bi bus
cy
cl
e
tv
bi
rd
co
sh w
ee
ch p
ai
r
ca
bo r
pe a
rs t
o
pl n
a
bo nt
ttl
e
1
vg
0.4
fa
ca
do t
pl g
an
ta e
b
ho le
rs
tr e
m ain
bi
ke
bi bu
cy s
cl
e
tv
bi
r
c d
sh ow
ee
ch p
ai
r
ca
bo r
pe a
rs t
o
pl n
a
bo nt
ttl
e
so
A
vg
cl
c e
sh ow
ee
p
bi
ch rd
ai
r
ca
bo r
pe a
rs t
o
pl n
a
bo nt
ttl
e
ca
so t
fa
do
ta g
b
tr le
ai
n
b
ho us
m rse
bi
pl ke
an
e
bi t
cy v
0.6
so
A
vg
0.8
CA1
CA2
SS
MCG
EB
SPA
A
ca
so t
fa
do
tr g
ai
n
bu
ta s
b
ho le
r
m se
b
bi ike
cy
cl
e
pl tv
an
e
co
w
bi
sh rd
ee
ch p
ai
r
pe ca
rs r
o
bo n
pl at
a
bo nt
ttl
e
A
1
1
0.8
0.6
CA1
CA2
SS
MCG
EB
SPA
(a) 1000 proposals, IoU = 0.8.
(b) 1000 proposals, IoU = 0.9.
1
0.8
0.6
CA1
CA2
SS
MCG
EB
SPA
(d) 2000 proposals, IoU = 0.9.
Figure 3. Comparisons of recall values for each of the 20 categories on PASCAL VOC 2012. The leftmost bars show the overall recalls.
1
0.8
0.8
0.6
0.6
0.4
0.2
0
0.5
Recall
Recall
1
CA1
CA2
SS
MCG
EB
0.6
0.4
0.2
0.7
0.8
0.9
0
0.5
1
1
0.8
0.8
0.6
0.6
0.4
0.2
0
0.5
CA1
CA2
SS
MCG
EB
0.6
0.4
0.2
0.7
0.8
(c) 1500 proposals per image.
0.6
0.7
0.8
0.9
1
0.9
1
(b) 1000 proposals per image.
1
Recall
Recall
(a) 500 proposals per image.
CA1
CA2
SS
MCG
EB
0.9
1
0
0.5
CA1
CA2
SS
MCG
EB
0.6
0.7
0.8
(d) 2000 proposals per image.
Figure 4. Recall versus IoU threshold on BSDS dataset.
1
0.8
0.8
0.6
0.6
Recall
Recall
1
0.4
CA1
CA2
SS
MCG
EB
0.2
0
0
500
1000
Number of proposals
1500
0.4
CA1
CA2
SS
MCG
EB
0.2
0
0
2000
500
1
0.8
0.8
0.6
0.6
0.4
CA1
CA2
SS
MCG
EB
0.2
0
0
500
1000
Number of proposals
1500
0.4
0.2
0
0
2000
500
0.6
1500
2000
(d) IoU = 0.8.
CA1
CA2
SS
MCG
EB
0.8
0.4
0.6
0.4
CA1
CA2
SS
MCG
EB
0.2
0.2
0
0
1000
Number of proposals
1
Area under recall
Recall
0.8
2000
CA1
CA2
SS
MCG
EB
(c) IoU = 0.7.
1
1500
(b) IoU = 0.6.
1
Recall
Recall
(a) IoU = 0.5.
1000
Number of proposals
500
1000
Number of proposals
(e) IoU = 0.9.
1500
2000
0
0
500
1000
Number of proposals
1500
2000
(f) Area under ”recall versus IoU threshold” curves.
Figure 5. (a)-(e) Recall versus candidate number curves on BSDS dataset, with IoU threshold ranged from 0.5 to 0.9. (f) Area under ”recall
versus IoU threshold” curves which are shown in Figure 4, for varying number of proposed windows.

Complexity-Adaptive Distance Metric for Object Proposals Generation

Transcription

Similar documents

Presentation