Untitled - AVLab.pl

Transcription

Untitled - AVLab.pl

PE
RF
ORMANCET
E
S
TOFF
RE
E
ANT
I
VI
RUSS
OF
T
WARE
ANDI
NT
E
RNE
TS
E
CURI
T
YS
UI
T
E
S
NOVE
MBE
R2014
Per
f
or
medt
estdat
e:
Oct
ober-November2014
Performance test – November 2014 / www.avlab.pl
INTRODUCTION…………………………...…………………………………………………………………………………1
TESTED PROGRAM VERSIONS…………………………………………………………………………………………..2
WHAT AND HOW WE TESTED…….………………………………………………………………………………………3
METHODOLOGY..………...…………………………………………………………………………………………………3
OTHER PRINCIPLES………………………………………………………………………………………………………...5
GRAPHS...……………………………………………………………………………………………………………………..6
HOW WE RATED……………………………………………………………………………………………………………16
CERTIFICATES GRANTED………………………………………………………………………………………………..17
As in last year as now we prepared for our readers performance test for the most popular paid and free security
software. Unfortunately during test pending from 20 October to 15 November several producers haven’t released
their new versions of software. We are talking about Czech TrustPort, which has released only TrustPort Internet
Security 2014 version unlike vast majority of competition. In similar situation is Polish Arcabit, which hasn’t
published its latest solution yet, as well as American Comodo, but we can expect stable version shortly, so we
decided to test BETA version of Comodo Internet Security 8.
Regarding test criteria this year we decided not to measure full system scan time. Such a thing is certainly
interesting, but we know that scanning should be above all effective. Regarding resources usage during scan, we
should take into consideration that some programs may use technologies under different names, which initiate
smart scan – when user isn’t using computer, scanning process receive more system resources.
Certainly many user noticed that antivirus programs can slow down operations such as archiving and data
extracting or copying and opening files. It’s because of on-access file scanning in real time during mentioned
operations. Antivirus programs use different solutions to reduce scan time, for example they don’t scan files twice
(only if they are modified) and files that don’t carry viruses. All similar options can be found in program settings.
Our advice for all antivirus software is as follows: perform full system scan immediately after installation which will
result in measurable benefits for user. Firstly you will see if system is clean, secondly antivirus program will scan
all files, so there is no need to use system resources later during work and entertainment, because most files will
be already scanned.
1
2
TESTED PROGRAM VERSIONS
Program name
Country
Version
Arcabit Internet Security 2014
Poland
2014/1411153813
avast! Free Antivirus 2015
Czech Republic
2015.10.0.2206
avast! Internet Security 2015
Czech Republic
2015.10.0.2206
AVG Antivirus Free 2015
Czech Republic
2015.0.5315
AVG Internet Security 2015
Czech Republic
2015.0.5315
Avira FreeA ntivirus 2015
Germany
14.07.306
Avira Internet Security 2015
Germany
14.07.306
Baidu Antivirus 5
China
5.0.3.84333
Bitdefender Antivirus Free Edition
Romania
1.0.21.10.99
Bitdefender Internet Security 2015
Romania
18.17.0.1227
BullGuard Internet Security 2015
United Kingdom
15.0.288.1
Check Point Zone Alarm Internet Security Suite 2015
USA
13.3.209.000
Comodo Internet Security 8 BETA
USA
8.0.0.4314
Dr.Web Security Space 10
Russia
10.0.0.10280
Emsisoft Internet Security 9
Austria
9.0.0.4570
eScan Internet Security with Cloud Security 14
India
14.0.1400
ESET Smart Security 8
Slovakia
8.0.304.2
FileMedic Internet Security 2014
Poland
17.21.0.925
Fortinet FortiClient 5
USA
5.2.1.0.605
F-Secure SAFE / Internet Security 2015
Finland
14.115.100
G Data Internet Security
Germany
25.0.2.2
Kaspersky Internet Security 2015
Russia
15.0.1.415
Lavasoft Ad-Aware Free Antivirus+
Malta
11.4.6792.0
Malwarebytes Anti-Malware Pro
USA
2.0.3.1025
McAfee Internet Security 2015
USA
13.6.1367
Microsoft Security Essentials
USA
4.0.6.305.0
Panda Free Antivirus 2015
Spain
15.0.4
Panda Internet Security 2015
Spain
15.0.4
Qihoo 360 Total Security 5
China
5.2.0.1066
Symantec Norton Security 2015
USA
22.0.1.110
Trend Micro Internet Security 2015
Japan
8.0.1133
TrustPort Internet Security 2014
Czech Republic
14.0.5.5273
Webroot SecureAnywhere Internet Security Plus
USA
8.0.5.107
WHAT AND HOW WE TESTED?
This year we took into consideration following aspects: system startup time with antivirus installed, CPU and
RAM usage during idle and scanning, copying time, archiving time, extracting time and program size after
installation including all virus signatures and files.
To verify all aspects mentioned above we prepared VMWare Workstation 9 on host //CPU: Athlon II X2 240
2,8GHz, //RAM: 2x4GB DDR3 1333 MHz, //HDD: Seagate 500 GB SATA II 7200 RPM //32MB Virtual Cache
Windows 7 x32 Professional + SP1 + critical updates dated 1 September (2 cores assigned, 2GB RAM) which we
were restoring after each test. In addition we installed following programs:
Windows Update has been disabled because of influence on the test results. Antivirus programs may use internet
connection and update their signatures.
Methodology
1. Restore system to initial state.
2. Download antivirus program from producer website and install with default settings.
3. Restart system and measure startup time first time.
3. Update signatures database and program files.
4. Restart system and measure startup time second time.
5. Check resources usage during idle and scanning.
6. Check copying time.
7. Check archiving and extracting time.
8. Check program size.
9. Restart system and third measurement, back to point 1.
3
System startup time
We used performance monitor to accurately measure system startup time, which is build into Windows. Time was
measured from selecting user account to full system start. Procedure was repeated three times: after program
installation, system was restarted, we measured time after second logging in, then we were updating signatures
and program files, system was restarted and after that we measure time again. Last measurement followed after
all other tests.
RAM and CPU
During idle to verify demand for resources by antivirus program, we checked usage of resources by its all
processes, not including process related to GUI, unless it automatically worked together with other processes.
After signatures and files update and system restart, we waited 2-3 minutes and after that we run performance
monitor. During data collection procedure, when program automatically was updating its signature or scanning
files, we aimed for smallest resources usage, so action was repeated couple of times. Results were collected
every second for 5 minutes, we calculated average value for each process, RAM memory and CPU was summed
which gave us accurate CPU and RAM usage by antivirus processes. Procedure of measurement during
scanning looked exactly the same with difference that we added process related to graphical user interface, that
was used to run full system scan. Data was also collected for 5 minutes.
Copying time
To see impact on data copying speed during on-access scan, we prepared folder on C partition 2,5GB in size
containing 120 files (ISO, EXE, JPG, ZIP, PNG, OTHER). Data from location above were copied to
C:\user\Desktop, time was measured with ExtremeCopy software.
Archiving and extracting time
To see impact on archiving and extracting process, we used Easy-7 ZIP. Prepared folder 320MB in size
contained 119 files was packed in zip and then unpacked. Time was measured with mentioned program.
Program size
Last thing is program size. Although all programs more or less use cloud technology, it doesn’t mean that in all
cases their size is reduced to minimum. This aspect wasn’t taken into consideration in the final result.
4
WE TAKE INTO CONSIDERATION…
All program were tested on one virtual machine, which had restorable system. This way, we don’t ignore caching
access to files by file system, so that each program had same testing environment. I most cases, antivirus
program scan files only once until they are modified. Special algorithm can judge, if file has been already
scanned and antivirus could have better result in our test. Running (without restoring clean system, even after
restart) scan on-access or on-demand causes second test can be finished much faster. Test performed that way
aren’t reliable, because the most important thing in this test is to measure drop in performance during scan on
clean files.
Scan time largely depends on process priority or relevant technology (local or cloud) which lies in fact that if user
is using computer during scanning, process priority is automatically limited not to slow down system.
Principle of equality
In comparative tests all programs should be tested on the same machine. Operation system and installed
software should be the same as well. Creating system or partition images is good way to map identical test
conditions.
Appropriate methodology
All performance measurements should be tested on “clean” files. Testing infected samples during scan may be
interesting, but such cases are extremely rare and most users don’t have hundreds or thousands infected files on
their hard drives. In addition, program settings should reflect the most common configuration, in most cases
default settings are the most appropriate solution.
Updates
Products should be fully updated before performance test (AMTSO recommendations), because some programs
may be installed without virus signatures, which may affect test results. Testers should be aware of Windows
Update and its potential impact in performance tests. Automatic system update should be disabled during test.
Update process may affect on research and changes made by it can lead to different results.
5
6
STARTUP TIME[ S ]
Startup 1
Windows 7 Professional x32 SP1
Startup 2
13
14
17
Webroot SecureAnywhere Antivirus 8
Startup 3
11
12
14
Average
13
15
14
15
18
15
18
17
15
17
18
15
17
17
18
15
20
18
19
19
18
18
19
20
19
17
22
19
19
17
18
18
18
17
18
19
19
18
25
18
16
19
22
20
16
19
18
19
17
19
20
20
20
20
21
19
20
20
Baidu Antivirus 5
23
17
21
20
22
20
20
20
20
19
23
18
14
23
22
24
19
17
20
21
21
28
18
21
20
24
20
21
22
21
21
21
Check Point ZoneAlarm Internet Security Suite 2015
22
22
23
24
19
22
24
21
21
22
27
25
Comodo Internet security 8 BETA
25
29
30
22
20
20
20
22
23
29
23
16
28
18
Avira Free Antivirus 2015
26
29
24
32
0
22
20
40
23
18
25
28
25
29
28
39
60
32
80
100
120
140
7
AVERAGE SYSTEM STARTUP TIME[ S ]
AV
0
Windows 7
5
10
15
20
15
17
17
18
18
19
19
19
19
19
19
20
20
Baidu Antivirus 5
20
20
20
20
21
21
21
21
22
22
22
35
15
18
30
13
25
22
23
23
25
25
28
32
8
CPU in idle [ % ]
0,005
0,010
0,031
0,036
0,037
0,047
0,047
0,048
0,058
0,063
0,063
0,063
0,074
0,084
0,092
0,114
0,126
0,154
0,167
0,189
Baidu Antivirus 5
0,192
0,203
0,220
0,235
0,348
0,364
0,464
0,502
0,565
0,830
0,850
5,235
9
RAM in idle [ MB ]
5
6
10
12
14
14
15
Baidu Antivirus 5
17
25
29
31
35
39
41
43
53
54
56
70
72
75
81
81
87
101
102
144
146
158
163
166
232
10
CPU scanning [ % ]
0,835
1,098
2,599
3,076
3,174
3,814
4,391
6,214
6,451
7,168
7,576
8,23
9,847
11,258
11,368
Baidu Antivirus 5
12,38
13,03
14,024
14,702
16,541
21,351
21,395
21,424
23,086
25,262
26,551
27,519
30,934
32,447
33,857
47,609
51,664
63,239
11
RAM scanning [ MB ]
10
17
18
25
29
37
48
53
57
Baidu Antivirus 5
59
60
70
78
81
92
101
106
108
113
118
123
144
178
180
180
182
184
188
191
198
219
232
268
12
Copying time [ s ]
190
191
195
197
199
200
202
207
209
231
236
238
242
248
248
Baidu Antivirus 5
252
254
255
256
258
261
267
270
279
279
285
288
303
303
322
323
324
330
337
13
Archiving [ s ]
23
24
25
25
27
27
29
33
33
34
34
35
35
36
36
37
Baidu Antivirus 5
37
37
38
38
40
40
43
44
44
46
47
47
48
50
50
68
75
160
14
Extracting [ s ]
8
10
10
11
12
12
13
14
15
15
16
16
16
16
17
18
18
19
20
21
21
23
23
Baidu Antivirus 5
25
32
33
37
41
43
51
54
59
62
68
15
Size after installation [ MB ]
1
30
48
56
95
96
97
108
112
118
130
130
139
147
151
158
Baidu Antivirus 5
207
211
214
310
345
364
370
394
402
417
443
455
567
568
578
579
812
Average system startup time, Windows 7: 13 seconds
13-16: 4 points
17-20: 3 points
21-24: 2 points
25-28: 1 points
29< : 0 points
Copying, Windows 7: 197 seconds
000-250: 4 points
251-300: 3 points
301-350: 2 points
351-400: 1 points
401<: 0 points
Archiving, Windows 7: 36 seconds
00-45: 4 points
46-54: 3 points
55-63: 2 points
64-72: 1 points
73<: 0 points
Extracting, Windows 7: 15 seconds
00-19: 4 points
20-23: 3 points
24-27: 2 points
28-31: 1 points
32<: 0 points
CPU [ % ] – idle and scan
00-10: 4 points
11-20: 3 points
21-30: 2 points
30-40: 1 points
41<: 0 points
RAM [ MB ] – idle and scan
00-75: 4 points
76-150: 3 points
151-225: 2 points
226-300: 1 points
301<
: 0 points
Final score
32-28: certificate Best+++
27-23: certificate Best++
22-18: certificate Good+
17-0: average quality
16
17
Short program name
Avg. CU*
Points
Copying
Points
Archiving
Points
Extracting
Points
CPU
Points
RAM
Points
CPU**
Points
RAM**
Points
Total
avast! IS 2015
20
3
231
4
44
4
16
4
0,22
4
29
4
4,391
4
37
4
31
AVG IS 2015
20
3
207
4
38
4
16
4
0,047
4
56
4
11,258
3
60
4
30
eScan IS with Cloud 14
21
2
191
4
27
4
8
4
0,114
4
12
4
1,098
4
25
4
30
360 TS 5
18
3
248
4
25
4
10
4
0,037
4
54
4
8,23
4
123
3
30
avast! Free2015
20
3
303
2
37
4
18
4
0,348
4
17
4
3,174
4
29
4
29
FortiClient 5
19
3
236
4
50
3
12
4
0,063
4
75
4
2,599
4
92
3
29
Panda Free 2015
18
3
254
3
23
4
19
4
0,092
4
35
4
9,847
4
78
3
29
Norton Security 2015
17
3
303
2
35
4
15
4
0,502
4
14
4
6,214
4
48
4
29
Comodo IS 8 BETA
23
2
261
3
47
3
13
4
0,364
4
10
4
7,168
4
17
4
28
FileMedic IS 2014
21
2
195
4
35
4
14
4
0,85
4
43
4
21,424
2
53
4
28
G Data IS 2015
19
3
267
3
36
3
23
4
0,058
4
31
4
11,368
4
101
3
28
Webroot SA 8
15
4
258
3
37
4
37
0
0,01
4
3
4
6,451
4
10
4
27
Avira Free 2015
28
0
238
4
43
4
16
4
0,235
4
15
4
3,076
4
178
2
26
Ad-Aware Free11
15
4
190
4
33
4
12
4
0,005
4
166
2
26,551
2
219
2
26
Panda IS 2015
21
2
209
4
25
4
33
0
0,047
4
14
4
7,576
4
57
4
26
Avira IS 2015
25
1
256
3
38
4
21
3
0,464
4
25
4
3,814
4
182
2
25
Baidu 5
20
3
252
3
37
4
41
0
0,192
4
41
4
12,38
3
59
4
25
Kaspersky IS 2015
19
3
330
2
68
1
17
4
0,031
4
72
4
16,541
4
108
3
25
18
MBAM Pro
17
3
200
4
24
4
16
4
0,126
4
102
3
33,857
1
268
1
24
McAfee IS 2015
18
3
242
4
29
4
18
4
0,063
4
144
3
63,239
0
198
2
24
TrustPort IS 2014
19
3
323
2
44
4
21
3
0,048
4
53
4
25,262
2
180
2
24
Bitdefender IS 2015
22
2
255
3
34
4
11
4
0,074
4
163
2
30,934
2
188
2
23
Dr.Web SS 10.0
25
1
322
2
46
3
10
4
5,235
4
81
3
21,395
3
118
3
23
Emsisoft IS 9
22
2
202
4
27
4
23
3
0,203
4
81
3
32,447
1
191
2
23
ESET SS 8
19
3
248
4
33
4
62
0
0,036
4
101
3
27,519
2
106
3
23
Trend Micro IS 2015
20
3
199
4
48
3
25
2
0,565
4
39
4
51,664
0
113
3
23
AVG Free 2015
21
2
270
3
50
3
54
0
0,167
4
70
4
13,03
3
81
3
22
F-Secure SAFE / IS 2015
19
3
279
3
40
4
32
0
0,063
4
87
3
21,351
2
144
3
22
Arcabit 2014
20
3
337
2
47
3
20
3
0,189
4
158
2
23,086
2
180
2
21
Bitdefender Free
22
2
279
3
160
0
43
0
0,154
4
6
4
14,702
3
70
4
20
ZoneAlarm IS
22
2
285
3
40
4
68
0
0,83
4
232
1
14,024
4
232
1
19
BullGuard IS 2015
23
2
288
3
34
4
51
0
0,084
4
146
3
47,609
0
184
2
18
Microsoft SE
32
0
324
2
75
0
59
0
0
4
5
4
0,835
4
18
4
18
*Average system startup time, **CPU and RAM during scanning.
19
32-28 Points
eScan Internet Security with Cloud 14
360 Total Security 5
Comodo Internet Security 8 BETA
G Data Internet Security 2015
27-23 Points
Webroot SecureAnywhere Internet Security Plus 8
BaiduAntivirus5
Dr.Web Security Space 10.0
22-18 Points
Check Point Zone Alarm Internet Security Suite 2015
---
Contact about tests for producers: [email protected]
Certificates granted for download in high resolution: http://avlab.pl/dla-prasy
About AVLab
AVLab brings together enthusiasts of antivirus software and web safety. Our activities include program testing
and sharing results of our analyzes with all internet users. We are not controlled and/or linked with software
producer in any way.
AVLab tests are independent and take place in conditions close to reality. Don’t be guided by our results when
making final decision in choosing antivirus program. In order to make final choice we suggest read tests from
other independent laboratories that use different methods and techniques for testing antivirus software. In
addition, decisions depends on personal preferences, specified features, efficiency, detection rate, impact on
system performance, user interface, price, usability, compatibility, language, technical support quality and many
other things.
20

Untitled - AVLab.pl

Transcription

Similar documents

pdf