Presentation - Professional Information Security Association

Transcription

Presented by
PISA Anti-Spam Project Group
3URMHFW*URXS0HPEHUV
Mr. John Tung (Leader)
Mr. Kent Kwan (Leader)
Mr. Billy Ngan
Mr. Howard Lau
Mr. Jason Luk
Ms. Lydia Chan
Mr. Manfred Hung
Mr. Raymond Tang
Mr. SC Leung
Mr. Wayne Tam
'LVFODLPHU
• This material is NOT intended to be adopted in the course
of attacking any computing system, nor does it encourage
such act.
• PISA takes no liability to any act of the user or damage
caused in making use of this report.
• The points made here are deliberately kept concise for the
purpose of presentation. If you require technical details
please refer to other technical references.
&RS\ULJKW
• The copyright of this material belongs to the
Professional Information Security Association (PISA).
• A third party could use this material for noncommercial purpose, given that no change in the
meaning or interpretation of the content was made and
reference is made to PISA. All rights are reserved by
PISA.
$JHQGD
1. Introduction to Spamming
•
Overview and Economy of Spams
2. Current Anti-Spam Solutions
1.
2.
3.
4.
5.
6.
Defense Approaches for Spams
Scan Spam Relay
Black-listing SPAM and Relay
Heuristic Content Filtering Development
Domain/Server Validation (Standard and Proprietary)
Some Proprietary Anti-SPAM features provisions
$JHQGD
3. Select Your Own Defense Strategy
1.
2.
3.
4.
End User Perspective
Corporate Perspective
ISP Perspective
Government
4. Cross Border Issues and Strategies
•
Legislation & Incident Response in a Global View
2YHUYLHZ DQG (FRQRP\ RI 63$0
Kent Kwan
2YHUYLHZ
What is spam?
• Unsolicited bulk mail is spam
• e.g. First contact enquiry or product
promotion sending to a large number of
recipients
• Commercial in nature
• Not requested by recipient
• Interpreted by recipient
2YHUYLHZ
• It wastes much of user time to handle
• Consumes Internet bandwidth and
storage space of mail server
• Lost productivity
• Can be malicious (i.e. Identify theft and
virus)
• For example: W32.Mimail.E And
W32.Sobig.E
2YHUYLHZ
• By HKISPA's survey, it revealed that
50% of all e-mail is spam (11 ISP are
involved and represent 90% of Internet
users in HK)
• Estimate to cost HK economy as much as
HK$10 billion per year (with HK$6
billion in productivity lost).
2YHUYLHZ
Why does it happen?
• Mainly for money
• Product advertising and Internet
marketing
• Take benefits from selling products
• Around US$250 per million e-mail sent
• Low cost in sending the messages to a huge
number of users
2YHUYLHZ
• Sending more mail, more profits
• A "good" spammer can easily earn US$100,000
per year. Spammers work on a piece rate so the
more spam they send the higher their income
potential. On average 1,000,000 pieces of junk
mail sent out will result in 150 "sales" or leads.
This in turn generally means big profits for the
spammer.
2YHUYLHZ
Who will be spammers?
• Currently around 180 well-known serious
spammers at work out there
• Technically competent and consider their
activities to be harmless
• Tendency to be involved in other illegal activities
(i.e. Credit card fraud)
• Will work with other spammers on large
campaigns
2YHUYLHZ
Strategies of spammers
• Playing tricks to attack attention
• Sender address is spoofed or forged, concealing
the identity of the spammer
• Using subject lines that have more insidious
appeal – e.g. “why haven't you reply”
• Greetings card scam – e.g. A message is waiting
for you in a linked web site
2YHUYLHZ
Current status of spamming worldwide
• More than 50% of email traffic are spam
• Estimate to cost US$59 billion annually in
the United State
• Most spam mail is from USA (58.4%),
and then China (5.8%) and UK (5.2%)
• It is rapidly increasing
Source:
www.messagelabs.com
Source:
www.brightmail.com
2YHUYLHZ
Useful information:
http://www.colinfahey.com/spam_topics/spam_the_phenomenon.htm
http://www.hkispa.org.hk/spam/20040113-coalition-paper.pdf
http://www.messagelabs.com/binaries/aspammerintheworks.pdf
'HIHQVH $SSURDFKHV IRU 6SDPV
Presented by Wayne Tam
6SDP 'HIHQVH $SSURDFKHV
Attack Strategies
• Direct SMTP
• Open Relay and Open
Proxy
• Oversea Relays when
control and regulation is
lax
• Fake Sender address and
domain
• Content (HTML, Scripts,
Language)
• Content Tricks
Defense Strategies
• SMTP Blacklist
• Open Relay Scan and
Blacklist; ISP block SMTP
• CERT collaboration and
legislation, ISP
empowerment
• Domain/server validation;
Sender validation
• Content filtering
• Trick filtering
6SDP 5HOD\ 6FDQ
Wayne Tam
:KDW LV DQ RSHQ UHOD\"
• A mail server that allows third parties to send mail to
other third parties.
• For example, the domain mydomain.com will accept
mail for users @mydomain.com from Internet users all
over the world;
• it also allows users on the machine to send mail to
Internet users all over the world. However, it does not
allow a user from, say, AOL.COM to send mail to a
user at, say, JUNO.COM. Doing that (which is a
popular technique used by spammers) is called a .thirdparty relay,. because the spammer is attempting to
relay the mail through mydomain.com.
:KDW LV D EOLQG RU DQRQ\PRXV RSHQ UHOD\"
• This means that the open relay replaces
the mail header with its own removing
details of who the message was sent from.
This enables spammers to send truly
anonymous spam.
&RPPRQ UHDVRQV IRU RSHQ UHOD\V
• Convenience (internal needs)
• Unpatched servers (default installations,
bugs)
• security vulnerability (e.g. worm, like
Bagle worm)
• mis-configuration
5HOD\ %DVHG EXON HPDLOHUV
1.
2.
3.
4.
Stealth,
Mach10,
SpeedyMailer,
Platinum Corporate Mailer
:K\ VKRXOG , ERWKHU VFDQQLQJ IRU RSHQ UHOD\V"
• If you don't aggressively close down open
relays in your network, then spammers will
find and abuse these servers. Spam puts an
unneeded strain on your network and mail
servers and is a pain to the millions of people
that find it in their inbox everyday. You will
also be likely to find your network on open
relay blacklists such as ORBS.
2UJDQL]DWLRQ FRPSLOLQJ EODFN OLVWV
• http://dnsbl.net.au/testing/
• Road Runner Probing :
http://sec.rr.com/probing.htm
• http://www.njabl.org Not Just Another
Bogus List
• America online aol.com email message
warning of test failure
• http://www.njabl.org/method.html
6RPH 5HDO7LPH %ODFN /LVWV
1. MAPS Realtime Blackhole List (http://mailabuse.org/rbl)
2. MAPS Relay Spam Stopper (http://mailabuse.org/rss)
3. MAPS Dialup User List (http://mailabuse.org/dul)
4. MAPS Open Proxy Stopper List (http://mailabuse.org/ops)
5. Spamhaus Block List
(http://www.spamhaus.org/sbl)
Don’t become a statistic!
7\SLFDO 7HVWV
• The 19 tests: (ref: www.linuxsec.net/Mail/openrelay/whitehats.com.mailrelay.html)
Relay test: #Test 1
>>> mail from: <[email protected]>
<<< 250 2.1.0 <[email protected]>... Sender ok
>>> rcpt to: <[email protected]>
<<< 550 5.7.1 <[email protected]>... Relaying denied
>>> rset
<<< 250 2.0.0 Reset state
Attempt to send email where source and destination addresses are the same.
7\SLFDO 7HVWV
Relay test: #Test 2
<<< 501 5.1.8 <[email protected]>... Sender domain must exist
>>> rset
Attempt to use an invalid source address.
7\SLFDO 7HVWV
Relay test: #test 3
>>> mail from: <spamtest@localhost>
<<< 553 5.5.4 <spamtest@localhost>... Real domain name required
>>> rset
Used the localhost hostname in the source address. This probably fools older
SMTP servers.
7\SLFDO 7HVWV
Relay test: #Test 4
>>> mail from: <spamtest>
<<< 553 5.5.4 <spamtest>... Domain name required
>>> rset
Omitted the domain name entirely, expecting that the mail would be
delivered as though it were local.
7\SLFDO 7HVWV
Relay test: #Test 5
>>> mail from: <>
<<< 250 2.1.0 <>... Sender ok
>>> rset
Omit source address entirely.
7\SLFDO 7HVWV
Relay test: #Test 6
>>> rset
Specify the FQDN (fully qualified domain name) of the victim server as the
host in the source address.
7\SLFDO 7HVWV
Relay test: #Test 7
>>> mail from: <spamtest@[23.23.23.23]>
<<< 250 2.1.0 <spamtest@[23.23.23.23]>... Sender ok
>>> rset
Use the IP address of the victim SMTP server enclosed in brackets.
7\SLFDO 7HVWV
Relay test: #Test 8
>>> rcpt to: <nobody%[email protected]>
<<< 550 5.7.1 <nobody%[email protected]>... Relaying
denied
>>> rset
Use % style relaying (legacy email systems may support this syntax).
7\SLFDO 7HVWV
• The 19 tests:(ref: www.linux-
sec.net/Mail/openrelay/whitehats.com.mailrelay.html)
Relay test: #Test 9
>>> rcpt to: <nobody%mail-abuse.org@[23.23.23.23]>
<<< 550 5.7.1 <nobody%mail-abuse.org@[23.23.23.23]>... Relaying
denied
>>> rset
Use & style relaying as well and using the victim SMTP server IP address
instead of it's FQDN.
7\SLFDO 7HVWV
• The 19 tests:(ref: www.linuxsec.net/Mail/openrelay/whitehats.com.mailrelay.html)
Relay test: #Test 10
>>> rcpt to: <"[email protected]">
<<< 550 5.7.1 <"[email protected]">... Relaying denied
>>> rset
Encapsulate the destination address in double quotes.
7\SLFDO 7HVWV
>>> rcpt to: <"nobody%mail-abuse.org">
<<< 550 5.7.1 <"nobody%mail-abuse.org">... Relaying denied
>>> rset
Use % style syntax and encapsulate in double quotes.
7\SLFDO 7HVWV
>>> rcpt to: <"[email protected]@test.whitehats.com">
<<< 550 5.7.1 <"[email protected]@test.whitehats.com">...
Relaying denied
>>> rset
Source email address hostname is IP of victim SMTP server, also destination
email uses @@ relay syntax and is enclosed in double quotes.
7\SLFDO 7HVWV
>>> rcpt to: <"[email protected]"@[23.23.23.23]>
<<< 550 5.7.1 <"[email protected]"@[23.23.23.23]>... Relaying
denied
>>> rset
Destination email address uses double quotes around the intended target,
and uses the IP address of the victim SMTP server.
7\SLFDO 7HVWV
>>> rcpt to: <[email protected]@[23.23.23.23]>
<<< 550 5.7.1 <[email protected]@[23.23.23.23]>... Relaying
denied
>>> rset
Relaying style without quotes but using the IP address of the victim SMTP
server.
7\SLFDO 7HVWV
>>> rcpt to: <@test.whitehats.com:[email protected]>
<<< 550 5.7.1 <@test.whitehats.com:[email protected]>...
Relaying denied
>>> rset
Another email syntax that may allow relaying.
7\SLFDO 7HVWV
>>> rcpt to: <@[23.23.23.23]:[email protected]>
<<< 550 5.7.1 <@[23.23.23.23]:[email protected]>... Relaying
denied
>>> rset
Another email syntax that may allow relaying, this time using the IP address
of the victim SMTP server.
7\SLFDO 7HVWV
>>> rcpt to: <mail-abuse.org!nobody>
<<< 550 5.1.1 <mail-abuse.org!nobody>... User unknown
>>> rset
Alternate email addressing syntax and IP address used in source
email.
7\SLFDO 7HVWV
Relay test: #test 18
>>> rcpt to: <mail-abuse.org!nobody@[23.23.23.23]>
<<< 550 5.7.1 <mail-abuse.org!nobody@[23.23.23.23]>... Relaying denied
>>> rset
Alternate email addressing syntax and victim SMTP server IP address used
in destination email.
7\SLFDO 7HVWV
Relay test: #test 19
>>> rset
Here they tried to use the "postmaster" source account name. Perhaps this
has a special significance for certain SMTP and will be permitted.
)UHH RQOLQH WHVW
1.
2.
3.
4.
5.
http://relay-test.mail-abuse.org
http://abuse.net/relay.html
http://members.iinet.net.au/~remmie/relay/
http://www.ordb.org/submit/
http://www.njabl.org/method.html
)UHH 7RROV
•
•
•
•
•
•
http://sourceforge.net/projects/smtprc/ ("C", FreeBSD only)
http://samspade.org (hardware failure, often slow!)
http://david.weekly.org/code/relaycheck.txt (Perl)
http://packetstormsecurity.nl/UNIX/scanners/relayck
.pl (Perl)
http://www.monkeys.com/mrt/ (Shareware) (Perl)
http://relayprobe.com/ (SMTP Server Hunter Shareware, US$149) (exe)
&RPPHUFLDO 6HUYLFHV
•
•
http://www.networkscanning.com/scan.php?family=S
MTP%20problems
http://security.effects.com/tests-checks/smtp.html
&RPPHUFLDO 5HOD\ 6FDQQHUV
•
Super Web Scan 8.0
http://www.brothersoft.com/Internet_EMail_Super_Webscan_20730.html
2SHQ 5HOD\ 6FDQ 'HPR
Author: Wayne Tam, 8/5/2004, for PISA
Brief Overview:
A mail relay on a (Linux) Redhat 9 Server, Sendmail
8.12.8 will be tested for its vulnerability as an open
relay.
Goal:
The demo here uses the Following to demonstrate
"technically" how easy it is to scan for open relay
using an automated tester
•
Approach:
A default installation of a Linux (Redhat 9) server with minimal
customized configuration is installed and
the MRT script is run on it on its own mail relay (i.e. 127.0.0.1) with
the default test message and test pattern.
After showcase of the movie (2 minutes); highlight the mail log file and
the files stuck
test done at 3:13pm 8/5/2004
DEMO TIME!
Remark: this demo cannot fully
reflect a real life scenario but
serves as a demo of the mail
relay tests process
DEMO TIME!
Result:
27 test messages sent
2 bounced message received
2 messages still stuck as of 4:25pm (deferred by yahoo.com) (2261,
2267)
13 messages ok received
6 messages caught - rejected at source
4 others - don't know where it went
/HJDOLW\ RI RSHQ UHOD\ VFDQ
Word of caution:
There are no discussion found specific to legality of
doing “open relay scan”. Discussions of a similar
issue – “Port Knocking” were found and
summarized below:
Analogy:
“port knocking” is analogous to “turning the knob of a
door lock” or “knocking on the windows” on
someone else house.
Opinions:
Would you shoot somebody knocking on your door?
Would you consider somebody knocking on your door and
windows has a malicious intent?
Would you consider somebody walking down the street,
turning the door lock on the doors of every house has an
malicious intent?
•
•
What if he/she is a security guard of the neighbourhood?
What if he/she is a security guard of the neighbourhood and
has advised you in advance and was agreed by you?
Bottom Line:
If you have an organizational responsibility to ensure the
security is upkept and your scan is agreed by your
organization’s authority – go ahead. Otherwise, not
recommended.
Even if you are authorized to do it, watch out for the surge
of traffic load.
:KDW VKRXOG , GR LI , ILQG DQ RSHQ UHOD\ LQ
P\ QHWZRUN"
• First of all you should try to contact the
customer and inform them that their mail
server is open to third party relaying. Tell them
that it needs to be secured immediately and
forward on the relevant documentation about
securing MTA.s. If the problem is not fixed
within 24hrs ask them to take the machine
offline until it no longer relays third party
email.
$SSHQGL[ 6RPH RI WKH 5HIHUHQFHV
1.
http://www.linux-sec.net/Mail/openrelay.gwif.html
2.
http://www.ordb.org/
3.
http://www.insecure.org
4.
http://irc.warg.co.uk/smtprc-website/faq.html
5.
http://dnsbl.net.au/testing/
6.
http://www.secwiz.com/Default.aspx?tabid=27 (a bunch of good scanners)
7.
http://david.weekly.org/code/relaycheck.txt (Perl)
8.
http://packetstormsecurity.nl/UNIX/scanners/relayck.pl (Epicurus) (Perl)
9.
http://www.monkeys.com/mrt/ (MRT) (Shareware, Perl)
10.
http://www.njabl.org/
11.
http://relayprobe.com/ (Shareware, US$149)
12.
How to test your system for open SMTP Relay
http://www.nwfusion.com/details/6398.html
13.
Statistical Analysis of Open Relay:
http://downloads.securityfocus.com/library/OpenRelay-analysis.pdf
%ODFNOLVWLQJ 63$0 DQG 5HOD\
John Tung
Prepared by
John Tung and Raymond Tang
'16%/
• DNSBL – DNS based blackhole list
• A black list of IP addresses and whole
networks
• It contains entries like spam sources,
open relays, open proxy, spam support
sites and DULs
• Check against source IP address with
DNSBLs in mail server level
'16%/ PHFKDQLVP
DNSBL
DNSBL
Server
Server
Internet
Spammer’s
Spammer’s
Mail
Mail Server
Server
Recipient’s
Recipient’s
Mail
Mail Server
Server
Spammer
Spammer
Recipient
Recipient
+RZ GRHV LW ZRUN" •
•
•
•
Based on DNS service
Implemented as a DNS zone file
Queried by simple and fast DNS lookups
Zone file example
23.4.134.100.dnsbl.example.com.
IN A 127.0.0.4
IN TXT “Spam identified”
+RZ GRHV LW ZRUN" • Inquiry mode: DNS query RBL server
for A record
• Originated IP in the list return 127.0.0.4
• Originated IP NOT in list return negative result
• Transfer mode: Copy entire RBL to your
hosts using DNS which will be updated
instantly whenever changes occur.
5%/ 6XUYLYDO
• Commercial RBLs like MAPS
Via
subscription
• Free RBLs like ORDB
Via
donation
:D\V WR JHW LQWR 5%/V
• Portscan for open proxy, open relay
• Honey pots email addresses posting in
usergroup and public forums
• Complaints by users, analyse spam
source
*HW RXW RI 5%/V
1) Make request by list owners
2) Correct any problems with the mail
server allowing for mail relay
3) Develop policies or a system that will
prevent future use of the mail server as a
spam source
4) Confirm with list owners with a list of
the above changes were made
5%/ LQ 86
• Mail Abuse Prevention System (MAPS)
• http://Mail-abuse.org
• Realtime Blackhole List (RBL)
Paid
subscription
List of networks to be used either to originate
or relay spam
5%/ LQ &KLQD
• Open RBL service called CBL
• http://anti-spam.org.cn
• Free subscription
5%/V RXWVLGH &KLQD
• Overkill – block whole class C address
without thorough checking
• Users outside China cannot receive email originated from China
• CBL works more prudent than
themselves
5%/ LQ (XURSH
• www.spamhaus.org
• Spamhaus Block List
SBL
(Spamhaus Block List)
• real-time blocklist of spam sources and spam
services
EBL
(Exploit Block List)
• real-time blocklist of illegal 3rd party proxies
Free
subscription
5%/ LQ +.
•
•
•
•
MailProve
www.mailprove.com
Paid subscription
Dedicated in Block Asian Language
Spam
• Founded in early 2003
• Target to be de-facto anti-spam service
provider in Asia
5%/ LQ -DSDQ
• Coordinated by Hart Computer Co. Ltd.
as a volunteer
• http://www.rbl.jp
• Services included:
Remove
host from RBL database
Third Party Relay Check
Black List Check
• Free of Charge
$QWLYLUXV 5%/ LQ -DSDQ
• Also, coordinated by Hart Computer Co.
Ltd. as a volunteer and launched at May
of 2004
• http://www.rbl.jp/virusrbl-e.html
• Provide function to prevent virus that
send copies of themselves by e-mail to
any e-mail address from infected
computer
$QWLYLUXV 5%/ LQ -DSDQ FRQW’G
• Similar to normal RBL, just configure the ail
server to do DNS query against “virus.rbl.jp”
• Enlisted by:
Received complaint by users and add into RBL after
analysis
RBL.JP group’s mail server automatically analyze the
virus-infected email and if necessary with some
manually checking before adding to RBL
• Free of Charge
5HIHUHQFHV
• http://www.rbl.jp
• http://korea.services.net
+HXULVWLF &RQWHQW )LOWHULQJ
'HYHORSPHQW
Lydia Chan
Prepared by Manfred Hung and Lydia Chan
(YROXWLRQ RI +HXULVWLFV
$QWLVSDP WHFKQRORJ\
check for keyword
case “mortgage”, score[money]=20;
case “$$$”, score[money]=30;
case “free”, score[money]=25;
case “xxx”, score[sex]40;
……
Keyword
Filtering
Score-based
Characters
Filtering
Statistical-based
Filtering
Neural
networking
&RQWHQW )LOWHULQJ
• Simple keyword filtering
Filter
the words that spammers usually use
eg. mortgage, money, special offer, stock, cheap,
free
Need to have rich set of all possible keyword
Spammer can predicate the filtering rule easily
False
False
Positive
Negative
Complexity
6FRUHEDVHG )LOWHULQJ
• Assign the likelihood of different
keyword/token based on experience
• Classify them into different category
• Define the threshold for different category
PR/media are less sensitivity
to adv. type mail
VS
Personal are welcome to
have job adv. mail
VS
Financial are sensitivity to
adv. type mail
Corporate don’t want to have
job adv. mail
6FRUHEDVHG )LOWHULQJ
• Train: based on IT experience to define the
score of different token
• Spammer can mis-lead the system easily
False
False
Positive
Negative
Complexity
%D\HVLDQ DOJRULWKP
• Base on Bayes Theorem (named after Thomas
Bayes), a conditional probability theory
• Algorithms used in voice, handwritten character,
face recognition, statistical science, etc.
• Calculate the probability of a message being spam
based on message characteristic (eg. subject,
content)
• Look for words (tokens) that are typical of spam.
Every token is assigned an individual score
• A spam score for the whole message is computed
from the individual scores
%D\HVLDQ DOJRULWKP
• Hash tables map tokens to
probability scores
eg. promotion=0.99,
guarantee=0.97625,
describe=0.01
Normal
SPAM
• Training with a bunch of
emails (spam & non-spam
mails)
False
False
Positive
Negative
Complexity
1HXUDO QHWZRUN
• Extract some characteristics from emails
Mail header
Sender e-mail address
Message content
URL link
MIME type
• Train the cell to identify spam and normal
emails
• Define the threshold level
affect false positive and false negative
False
False
Positive
Negative
Complexity
0DLO PHVVDJH VDPSOLQJ
•
•
•
•
•
Required for training
Collected by honeypot mail system
Un-used e-mail address – honeytoken
ISP reported spam message
End-user reported spam message
Mixed of
spam &
non-spam
Able to
identify
spam
Feedback
Engine
Training
&RQWHQW 7ULFNV
• Spammer can still have many content tricks
Obfuscation
Invisible Ink
Small characters
Image
Encoded URL
Fragment words with <table> tags
Many other tricks…
&RQWHQW 7ULFNV – 2EIXVFDWLRQ
• Uses keyword filtering by obfuscation, eg. doll-ars, (mo-rt.gage)
• Replace letters that look like numbers with
numbers, e.g. m0rtg4ge
• Use accented characters in English
• Split words using HTML comments
e.g. mortgage
Mort<b></b>gage
• Insert small fonts characters in the spam
word
&RQWHQW 7ULFNV 2EIXVFDWLRQ
&RQWHQW 7ULFNV – ,QYLVLEOH ,QN
&RQWHQW 7ULFNV – 6PDOO FKDUDFWHUV
&RQWHQW 7ULFNV – 6PDOO FKDUDFWHUV
&RQWHQW 7ULFNV – 7H[W +70/
• Two part MIME
document with
the spam message
in the HTML
section and bogus
text in plain text
section.
&RQWHQW 7ULFNV – ,PDJH ZLWKRXW ZRUGV
&RQWHQW 7ULFNV – )DNH PDLO KHDGHU
%HWWHU 6ROXWLRQ
Cocktail Approach
Bayesian filter + Content filter heuristics against tricks
• Recognize the differences between human and machine
pattern matchers
• Require heuristics on content filtering & other spam
characteristics
• Should be applied in email header, subject line, body,
including text, html and scripts, etc.
• Combining evidence for spam identification
%D\HVLDQ +HXULVWLFV
• Token
not only words
but also other typical characteristics of spam
include alphanumeric characters, dashes, apostrophes
and dollar signs
• Score
assigned according to characteristic element for the
whole message
• Use fix no. of most interested tokens (eg. 20)
instead of all tokens
%D\HVLDQ +HXULVWLFV
• Degeneration: if you can’t find an exact match for a
token, treat it as if it were a less specific version
e.g.. if can’t look for Subject*free!, it can look for Subject*free,
free!, free
• Take care some of case sensitive
e.g. Act=0.98 and act=0.62
• Words that are mis-spelled or broken up have
higher scores
• Filter not only based on individual words, but also
on word pairs or even triples
e.g. offers=0.96, special offers=0.99, it offers=0.1
%D\HVLDQ +HXULVWLFV
• Probabilities - calculated individual user or division
or organization
• Can be trained by user simply to spam or not-spam
• For deployment in server
mail collected at hundreds of email addresses because
characteristics of good mail are different for different persons
• Recognizing non-spam features is more important
than recognizing spam features to lower the rate of
false positive
look for characteristics of legitimate mail
lower the complete score to protect from identifying
legitimate email as spam
2WKHU +HXULVWLFV RQ FRQWHQW ILOWHU
• Specific spam feature such as not seeing
recipient’s address in the To: field
• Include URL (esp. IP address), phone no, mail
address
• Send out a crawler to look at the included URL
site
• Accumulate a giant corpus of spam from huge no.
of users
• Ignore html comments and some html tags
• Cater for mail contain images by searching
“Href”, “img”, image file name together with URL
6RPH )LOWHUV
• Some Bayesian filters:
SpamBayes, SpamPal, Bogofilter, POPFile,
SpamProbe, Spamassassin (v2.5 or above), Spam
Bully, SpamTUNNEL, Python Anti-Spam Proxy
(PASP), Spamrunner, Death2spam (Commercial)
• Other algorithm: Markovians (CRM114)
• Reference:
http://crm114.sourceforge.net/Plateau99.ppt
6DPSOH 6SDP3DO
6DPSOH 6SDP3DO
From: "Cyrus Bird" <[email protected]>
To: [email protected]
Subject: Valium now offered at HUGE savings.
Date: Mon, 19 Apr 2004 02:36:04 -0100
X-Mailer: freeze erwin
flatworm-elapse: of viscosity notorious
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="--397394892899387"
X-Bayesian-Result: Spam (99)
X-Bayesian-Words: 0066ff 99 066ff 99 13px 95 abel 0 amend 0 bird 0
courage 99 curve 98 etica 99 gif 99 lvetica 99 packaging 97 price 97
savings 98 textbook 0
X-SpamPal: SPAM BAYESIAN_PLUGIN BODY
6DPSOH 6SDP3DO
Subject: Expand your business horizons at Microsoft Solutions Day! **SPAM
BAYESIAN_PLUGIN BODY**
Date: July 09, 2004 AM 4:00
Mime-Version: 1.0
Content-Type: text/html; charset=us-ascii
X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on
yellow.alumni.cuhk.edu.hk
X-Spam-Level: **
X-Spam-Status: No, hits=2.5 required=5.0
tests=FRONTPAGE,HTML_FONTCOLOR_RED,
HTML_FONTCOLOR_UNKNOWN,HTML_FONTCOLOR_UNSAFE,HTML_MESSA
GE,
LOTS_OF_STUFF,MIME_HTML_ONLY,NORMAL_HTTP_TO_IP autolearn=no
version=2.63
X-Bayesian-Result: Spam (100)
X-Bayesian-Words: 0in 0 140 99 comic 3 coupon 99 emailer 99 entrance 2 grand 1 harbour
99 icon-info 99 jpg 99 locally 2 mso-cellspacing 99 mso-no-proof 0 now 99 why 99
X-HTMLModify: CLEAN - meta-tags removed
X-SpamPal: SPAM BAYESIAN_PLUGIN BODY
5HIHUHQFHV
•
•
•
•
http://www.paulgraham.com/spam.html
http://crm114.sourceforge.net/
http://www.jgc.org/
http://email.about.com/cs/bayesianfilters/
a/bayesian_filter.htm
• http://home.dataparty.no/kristian/review
s/bayesian/index.php
2.5
Domain Validation Technology
(Standard and Proprietary)
Kent Kwan
Prepared by Jason Luk, Howard Lau and
Kent Kwan
Domain Validation 7HFKQRORJLHV
6WDQGDUG
• Reverse DNS Lookup
3URSULHWDU\
• AOL
SPF since Jan 2004
• Yahoo
Domain keys
• Hotmail
Caller ID
5HYHUVH '16 /RRNXS
•
When your server sends a "HELO" message to the
receiving server and that message contains your
domain name. The SMTP connection tells the receiving
server your current IP address
•
The receiving server compares the domain name it
discovered through performing the "reverse lookup"
(reverse PTR record) with the domain name your
server sent. If these two domain names are not the
same, the receiving server assumes that the email
message is SPAM and rejects it.
5HYHUVH '16 /RRNXS
•
•
•
Not everybody have reverse PTR's for their
mail servers
Some companies or individuals who have their
own mail server operating on a dynamic IP
address encounter problem
Another downside with reverse DNS is that you
have to make a DNS lookup with every email.
Using reverse DNS it slows down mail
transactions and further chokes internet
bandwidth under high load.
6HQGHU 3ROLF\ )UDPHZRUN 63)
• SPF allows internet domain administrators to
describe their e-mail servers in an SPF record that
is attached to the DNS (Domain Name System)
record.
• Other internet domains can then reject any
messages that claim to come from that domain but
were not sent from an approved server
SPF record
SPF record
Mx1.abc.com.hk. IN TXT "v=spf1 a -all"
Mx2.abc.com.hk. IN TXT "v=spf1 a -all"
mx3.abc.com.hk. IN TXT "v=spf1 a -all"
Ref: http://spf.pobox.com
Yahoo – DomainKeys
Sending Servers
Private key
Public key
Step A: Setup
• Domain owner generates a public/private
key pair to use for signing all outgoing
messages (multiple key pairs are allowed).
• Public key published in the DNS
• Private key stored in outbound email
servers.
Step B: Signing
• When an email is sent by an authorized
end-user, the outgoing email system uses
the private key to generate a digital
signature of the message.
• The signature is pre-pended to the header
of the email. The email is sent to the
Source: http://antispam.yahoo.com/domainkeys
recipient.
Yahoo – DomainKeys
Receiving Servers
Public key
Step C: Preparing
•
The DomainKeys-enabled receiving email
system extracts the signature and claimed
From: domain from the email headers
•
It fetches the public key from DNS for the
claimed From: domain.
•
Verifying:
The receiving email system uses the public
key from DNS to verify that the signature
was generated by the matching private key.
This verifies both the email sender and the
integrity of the email content.
Step D: Delivering
•
Source: http://antispam.yahoo.com/domainkeys
If verified and anti-spam tests don't catch
it, the email delivered to the user's inbox;
else it is dropped, flagged, or quarantined.
Microsoft - Caller ID
Two key steps
1.
Senders of e-mail publish the IP addresses of their outgoing
mail servers in DNS in an e-mail policy document.
2.
The e-mail software at the receiving end of a message
queries DNS for the e-mail policy and determines the
"purported responsible domain" of the message. This is
done by comparing the information in DNS to ensure it
matches the information on the originating mail.
63) 96 &DOOHU ,'
Both requires e-mail senders to modify DNS to declare
which servers can send mail from a particular internet
domain.
SPF only allows receiving domains to verify the "bounce
back" address in an e-mail's envelope, which is sent before
the body of a message is received and tells the receiving email server where to send rejection notices.
The "from" address checked by Caller ID is often a more
accurate indicator of the message's origin than the bounce
address
6HQGHU,' &DOOHU,' 63) PHUJHG
• Microsoft has submitted a draft technical
specification of the e-mail authentication system
Sender ID to the Internet Engineering Task Force
(IETF) for consideration as an industry-wide
standard.
• Sender ID combines Microsoft's Caller ID for Email (which was submitted to the IETF for
consideration in May 2004) with Sender Policy
Framework (SPF).
<DKRR VXEPLW 'RPDLQ.H\V WR ,(7)
•
•
Yahoo submitted a draft for DomainKeys to the IETF
standards body to begin the standardisation process.
DomainKeys works differently than Caller ID and SPF
•
using encryption to generate a signature based on the e-mail message
text that is placed in the message header
It is believed that Yahoo's technology is more secure than Caller ID and
SPF, because even if an e-mail message gets forwarded across various
e-mail servers, it's signature stays intact, allowing the receiving system
to verify its origin.
While DomainKeys is a better long-term fix for the spam
problem, Caller ID and SPF - or a merged standard - have
the advantage of being light-weight and easy to implement,
while closing many of the technical loopholes exploited by
spammers.
2.6
Review of some proprietary AntiSPAM features provisions
Kent Kwan
$2/ – IHDWXUHV RYHUYLHZ
• Block by word, subject and sender
Set filter to reponsed to certain word. Email containing
these cues are delivered to spam folder
• Filtering adapts to your preferences.
After you mark unwanted messages, your filters
remember and reroute junk e-mail automatically.
• Sort out mail from unknown senders.
Opt to only see e-mail from Address Book or Buddy
List® contacts in your inbox.
$2/ – IHDWXUHV RYHUYLHZ FRQW
• Skip certain Web addresses
Block out marketing or other unwanted messages from
certain Web sites.
• Block images and URLs.
Filtering tools help you prevent images or Web links
from displaying in e-mail from unknown senders.
• Bring buddies to the top.
Automatically pull messages from people on your Buddy
List® feature to the top of your mailbox.
6ROXWLRQV E\ VHUYLFH SURYLGHUV – +RWPDLO •
Microsoft uses many of its antispam technologies in Hotmail
and MSN
Intelligent Message Filter for Exchange, a spam filter based on
SmartScreen, for the Exchange e-mail server product.
One proofing technology that Microsoft is working on sends a challenge in
the form of a computational puzzle to the sender of a message if the filtering
system suspects a message may be spam. The sender, or the sender's
computer, would have to solve the puzzle to validate the legitimacy of an email message. Solving a challenge would take little time for a regular e-mail
sender's computer but would overwhelm the computing cycles of someone
sending large amounts e-mail. The technology is now being developed and
should be ready within a year
Microsoft's SmartScreen filtering technology, a sender authentication
technology called Caller ID for E-mail and "white lists" that contain
certified e-mail senders.
6ROXWLRQV E\ VHUYLFH SURYLGHUV – +RWPDLO •
•
•
Microsoft use IronPort Systems (messaging appliance supplier) to
bring new antispam capabilities for MSN.com
Bonded Sender is an IronPort program that identifies legitimate
commercial e-mail senders that have put up a financial bond and
adhere to rules about how they will send mail to recipients.
The program addresses the problem of spam filters falsely identifying
legitimate commercial e-mail as spam and deleting it.
Microsoft will allow suppliers that are registered and preapproved in IronPort's
Bonded Sender program to send e-mails to Hotmail and MSN users without the
messages being subject to normal Hotmail or MSN antispam controls.
Recipients can then choose to receive e-mails from the bonded senders by
opting in for the commercial e-mail.
Microsoft is the latest of about 28,000 ISPs, universities and corporations that
have signed up to receive bonded sender mail after testing the program for
about five months (As of May 2004)
6ROXWLRQV E\ VHUYLFH SURYLGHUV –<DKRR •
•
Yahoo free email service provide “SpamGuard” service
Yahoo paid email service (Yahoo Mail Plus) is with 4 antispam
enhancements
AddressGuard, a feature which gives Yahoo Mail Plus users disposable e-mail
aliases they can use instead of their real e-mail address.
•
For example, a user might want to use an alias when conducting an online transaction
or subscribing to an online group. If the alias then falls into a spammer's hands, the
user can simply ditch this disposable address.
A user can have up to 500 e-mail aliases in use simultaneously. E-mail sent to
these addresses arrives in the user's inbox. (This is not available to users of
Yahoo's free e-mail service).
SpamGuard Plus, an enhancement to Yahoo's proprietary SpamGuard spam filter.
SpamGuard Plus lets users customise the general SpamGuard filter based on their
usage and preferences. (This is only available to Yahoo Mail Plus users).
Message Views, which lets users sort e-mail sent by people listed in their
personal address book and by unknown senders.
•
This could help a user prioritise which messages he reads and replies to first, the logic
being he would want to deal first with e-mail from senders he knows. (This is available
to all users of Yahoo's e-mail service.)
6ROXWLRQV E\ VHUYLFH SURYLGHUV –<DKRR • Yahoo decided to invest in AntiSpam resource centre
because it has found out through surveys that
many users still ignore the basics of how to protect
themselves from spam, e.g. not replying to
unsolicited e-mail messages.
All Yahoo mail users have access to this resource centre.
A section of Yahoo's website network with information, tips
and best practices on dealing with spam.
Policy of Service Providers
AOL Unsolicited Bulk E-mail Policy
• http://postmaster.aol.com/guidelines/bulk_email.html
Yahoo! Universal Anti-Spam Policy
• http://docs.yahoo.com/info/guidelines/spam.html
Yahoo! Universal Anti-Spam Policy
• http://privacy.msn.com/anti-spam
$OOLDQFH RI 6HUYLFH 3URYLGHUV
• The Anti-Spam Technical Alliance
best practices and technical recommendations for the
industry
Adopting companies:
• Yahoo, Microsoft, AOL, Earthlink
http://docs.yahoo.com/docs/pr/release1169.html
5HIHUHQFH
•
Yahoo
•
•
•
•
ISP
•
•
www.hkispa.org.hk
Hotmail
•
•
•
http://www.computerweekly.com/articles/article.asp?liArticleID=125829&liFlavourID=1&sp=1
http://www.computerweekly.com/articles/article.asp?liArticleID=120098&liArticleTypeID=1&liCa
tegoryID=2&liChannelID=20&liFlavourID=1&sSearch=&nPage=1
http://antispam.yahoo.com/domainkeys
SPF & Caller ID
•
•
•
http://www.microsoft.com/mscorp/twc/privacy/spam_senderid.mspx
8VHU
(QG 3HUVSHFWLYHV
SC Leung & Billy Ngun
6WUDWHJLHV
1. Validate the Sender
2. Trace the Spam Source and report to
ISP
3. Filter by external tool/service
4. Filter by your own tool
POP3 Proxy, e.g. SpamPal
Outlook Plug-in, e.g. SpamBayes
9DOLGDWH 6HQGHU
• S/MIME (PKI)
• PGP (Ring of Trust)
• Pro
universal
validate both sender and content of
message
• Con
requires sender & receiver action for
each email
7UDFH 6SDP 6RXUFH DQG UHSRUW
• Viewing mail header in different mail programs
Reference: http://www.spampal.org/usermanual/headers_guide.htm
e.g. Outlook: at mail message, choose “View | Option” from menu
Return-Path: <[email protected]>
X-Original-To: [email protected]
Delivered-To: [email protected]
Received: from mail.abc.com (yellow.alumni.cuhk.edu.hk [127.0.0.1])
by mail.abc.com.hk (Postfix) with ESMTP id AFB783EC514;
Sat, 24 Jul 2004 23:46:48 +0800 (HKT)
Received: from plns-216-222-225-137-pppoe.dsl.plns.epix.net (plns-216-222-225-137-pppoe.dsl.plns.epix.net [216.222.225.137])
by mail.abc.com (Postfix) with SMTP id 7B8673EC509;
Sat, 24 Jul 2004 23:46:46 +0800 (HKT)
X-Message-Info: %XMESSB
Received: %RECB
%BY
Sat, 24 Jul 2004 20:41:46 +0400
Message-ID: %MESSIDB
Reply-To: "Sammy Werner" <[email protected]>
From: "Sammy Werner" <[email protected]>
To: [email protected]
Cc: [email protected], [email protected]
Subject: Stock Market Standout? **SPAM BAYESIAN_PLUGIN BODY**
Date: Sat, 24 Jul 2004 20:44:46 +0400
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="--84220928498948505002"
7UDFH 6SDP 6RXUFH DQG UHSRUW
• Read the header of
mail by freeware
SamSpade
http://www.samspade.org/ssw/
)LOWHU E\ H[WHUQDO VHUYLFH
• Forward email to *Yahoo!* which has
SpamGuard filter (free-of-charge)
• Read filtered email from Yahoo!
YahooPOPs! for Win (GPL) allows you to read Yahoo!
mail back to your POP3 email client, e.g. Outlook
http://yahoopops.sourceforge.net/
Version 0.6 (May 2004)
Can Send/Receive email using Yahoo! account
Has SSL option
Remark: work for personal email account only
<DKRR WDJJHG – ;5RFNHW6SDP
• X-YahooPOPs-Folder: @B@Bulk
• X-Apparently-To: [email protected] via
206.190.38.237; Fri, 09 Jul 2004 20:47:22 -0700
• X-Rocket-Spam: 69.42.121.107
• X-YahooFilteredBulk: 69.42.121.107
• X-Originating-IP: [69.42.121.107]
• Return-Path:
<[email protected]
om>
• Received: from 69.42.121.107 (HELO svr107.exhilaratedassistance.com) (69.42.121.107)
)LOWHUE\ \RXURZQ WRRO
• e.g. MS Outlook
Rule Wizard
Cons: works with
keyword only, not
intelligent enough
6SDP3DO
• Platform
Specially for Windows platforms (Win95 or +)
• Unix should use Spam Assassin
POP3 or IMAP4 mailbox
Email client (Outlook, OE, Eudora, etc.)
• CANNOT work with proprietary email like Yahoo, MSN,
AOL
• Author
James Farmer
:RUNLQJ 0RGHO
• POP3 Proxy
SpamPal
POP3
request
POP3
request
POP3 Proxy
localhost:110
Mail
Mail Client
Client
analyzed
Mail (tagged)
Spam Classifier
Client Local Machine
incoming
mail
POP3
POP3 Server
Server
pop.domain.com:110
pop.domain.com:110
&RQILJXUDWLRQ
• Change made to email clients, e.g. Outlook
Normal Config.
Config. for SpamPal
Incoming Mail Server (POP3):
pop.domain.com
localhost
User Name:
userid
[email protected]
:KDW LV 6SDP3DO"
• Mail Classifier
Sit between mail client and mail server, intercept all
emails
Tag spam email with header information. Handling is
left to the email client
• POP3/IMAP Proxy
Work for localhost
but can be configured to work as a server – mind the
performance and usability as a server
&ODVVLILHUV &RFNWDLO $SSURDFK
• Blacklists
• Content Filters
DNSRBL public blacklists
IP addresses
Email addresses
Country
• Exceptions to Blacklists
Bad Word
Bayesian Plug-in
URLBody Plug-in
HTML Modify Plug-in
Ignorelist
• Ignore blacklist for certain
providers, e.g. Yahoo, …
Whitelist
• Email addresses
• Automatic when receiving
non-spam in ## separate
days
$GGLQJ ([WUD '16%/ 'HILQLWLRQV
1. Click “Extra DNSBL Definitions” button
2. Place new definition to the end of extra_dnsbl.txt file:
LIST PSBL
NAME Passive Spam Block List
WEBSITE http://psbl.surriel.com/
ZONE psbl.surriel.com
DESCRIPTION An easy-on, easy-off blacklist that doesn't rely on testing
and should reduce false positives
RESULT_CODE 127.0.0.2 # Your server sent spam to trap-server recently
3. Save it, click OK to dismiss the SpamPal options window
then open it again - Passive Spam Block List should now be
listed with the other blacklists.
Reference: http://www.spampal.org/usermanual/optimize.htm#i35
7DJJLQJ 0HWKRG ,
• Mark at Subject Line
7DJJLQJ 0HWKRG ,,
• Mark at Mail header
6WDWXV DQG 3HUIRUPDQFH
• Statistics of Spam vs Passed
• False Positive of various RBL and country blocking
%D\HVLDQ )LOWHU 7UDLQLQJ
• Reclassify Feedback by user
2XWORRN 3OXJLQ 6SDP%D\HV
• SpamBayes
An open source project
Outlook add-in with Python-based SpamBayes engine
Written by Python hacker Mark Hammond
It’s free
• Platform
• For Outlook add-in: Windows and Outlook 2000 and XP
2XWORRN 3OXJLQ 6SDP%D\HV
• SpamBayes provides
a spam filter based on statistical analysis of personal mail
Train on your own unique message database
Learn by positive and negative clues
• SpamBayes appears
Toolbar called Anti-spam
+RZ 6SDP%D\HV ZRUN"
• No built-in rule, require training
• 2 ways for training, gather the clues
SpamBayes
initially consider all mail items
“unsure”
Presorting mail into two folders
• One containing only examples of good messages
• and another containing only examples of spam
+RZ 6SDP%D\HV ZRUN"
• Spam score given as message arrived
100%
being certain spam
0% being certainly not spam
SpamBayes use score to classify mail into one
of three categories – certain spam, unsure, and
good messages
+RZ 6SDP%D\HV ZRUN"
• 3 ways in which SpamBayes get things
wrong
A
spam stays in “Inbox” folder (False negative)
Any message is moved to “unsure” folder
A good message is moved to “spam” folder
(False positive)
+RZ 6SDP%D\HV ZRUN"
• Any message is moved to “unsure” folder
+RZ 6SDP%D\HV ZRUN"
• A good message is moved to “spam” folder
&RPSDULVRQ ZLWK RWKHU ILOWHUV
• POP3 Proxy vs. Mail Client Plug-in approaches
POP3 Proxy: SpamPal, SpamAssasin
• More neutral to technology, more transparent
• Possibility of a public proxy
• More difficult to configure
Mail Client Plugin: SPAMBayes
•
•
•
•
Integrate better with email client and easy to use
Performance hog to email client observable
Not possible as a public proxy
SPAMBayes needs time to train
&RPSDULVRQ EHWZHHQ 6SDP3DO DQG
6SDP$VVDVLQ
• Comparison between SpamPal and
SpamAssasin
Perl:
SpamAssasin uses Perl, works in Linux
and Windows. SpamPal is a windows service.
Multiple POP3 servers handling:
SpamAssasin uses 1 port per server (110, etc).
SpamPal opens one port (110) only, making use
of email client “userID” field to identify the
mail server.
&RUSRUDWH 3HUVSHFWLYH
Howard Lau
Prepared by Kent Kwan
• Corporate: Gateway side protection
Network
layer protection
Blocking by IP address
Reverse DNS lookups
TCP/IP connection limits
Disable open relay
• Real Time Blocklists (RTB)
RTB
are operated outside of an organization
control.
• User defined whitelists
• Sender Policy Framework (SPF)
Verifying
"MAIL From:" (envelope
originator) domains with the DNS.
Expensive
content-based spam checks can
be bypassed, saving resources on the
receiver side.
DNS loading increase
Use trusted DNS
*DWHZD\ VLGH SURWHFWLRQ FRQW
• Content filtering
• For users cannot risk missing any mail,
tag the message and deliver message to
user
• Base on techonologies such as heuristics
and textural analysis
*DWHZD\ VLGH SURWHFWLRQ FRQW
• RFC2505
13 recommendations for SMTP MTAs
•
http://www.ietf.org/rfc/rfc2505.txt
• SPF Filter – freeware for Windows
•
http://www.michaelbrumm.com/smtpspffilter.html
,63 3HUVSHFWLYH
Howard Lau
,63 3HUVSHFWLYH
• Technical
Mail servers shall not be allowed to relay mail from third parties
There shall be a restriction on the amount of outgoing mail
provided for web e-mail and pre-paid accounts
All clients using switched access shall not have outgoing TCP
access to the Internet on port 25 (SMTP). An SMTP server shall be
provided by such accounts; if possible the users outgoing SMTP
connection will automatically be redirected to such server
Realtime-Blackhole-List
Incoming SPAM Filtering
Limit NNTP Postings
Mailing Lists
,63 3HUVSHFWLYH
• Technical
Controlling
Customer Action (Rate Limiting)
• Control the max. no. of “TO:” recipients, e.g. iCable
• “Sendmail Outgoing SPAM Filter” provides a
similar function
(http://www.chel.com.ru/~anton/projects/spam_filter
/)
• Use of Throttle mail server
(http://www.templetons.com/brad/spam/block.html).
All mail delayed in queue for 1-2 minutes. If an IP
reaches a threshold, their mail are slowed
down/stopped
,63 3HUVSHFWLYH
• Administrative
There shall be an 'abuse' account. Mail sent to this account shall be
routed to a responsible person or team who has the ability to
investigate and take action on such complaints
All complaints sent to 'abuse' shall be replied to. This may be via
auto-responder
Complaints shall be investigated and action must be taken against
users flouting the Terms and Conditions referring to SPAM. Even
if investigation reveals no faults from ISP or user, the ISP is
encouraged to help the complainant to resolve their complaint
• Others
Terms & Conditions with Subscribers
• The subscriber shall not engage in sending SPAM messages.
Otherwise, it results in closure of the subscriber's account
• The definition of SPAM shall be stated
*RYHUQPHQW 3HUVSHFWLYH
SC Leung
6WUDWHJLHV DW 3RLQWV RI &RQWURO
Proactive,
Proactive,
Collaborative
Collaborative
- Accountability
- Enforcement
- Block
- Rate Limit
- Filter
Reactive,
Reactive,
Independent
Independent
- Accountability
- Enforcement
- Block
- Rate Limit
Corp
Corp
MTA
MTA
- Block
- Rate Limit
ISP
ISP
MTA
MTA
- Block
- Block
- Block
- Filter on demand
- Filter
- Filter
ISP
ISP
MTA
MTA
MTAs
MTAs
Corp
Corp
MTA
MTA
Recipient
Recipient
Sender
Sender
Source
Intermediate
Destination
MTA: Mail Transfer Agent
Insight from http://www.brandenburg.com/presentations/spamtechconsider.ppt
*RYHUQPHQW “3HUVSHFWLYH”
• Accountability
Sender/ Sending host
• Enforcement
Pros of Enforcement by Law
•
•
•
•
•
Contract (T&C between ISP and
Sender)
Regulation (on ISP by OFTA)
•
Law (on ISP and Sender by
Court)
•
•
• Scope of Control (Opt-in vs.
Opt-out; marketing vs.
anonymity etc.)
Proactive
Less false positive
Save bandwidth
Held the accountable
More effective than Code of
Practice
Cons
•
Intervention, discourage selfregulation
Side Effect may arise (scope)
Cross border collaboration is
needed
Takes long time to develop
/RFDO /HJLVODWLRQ DQG 5HJXODWLRQ
• Currently there is no law punishing spamming
• HKSARG is consulting on the regulation of
unsolicited message (e-mail, mail and fax)
http://www.ofta.gov.hk/report-paperguide/paper/consultation/20040625.pdf
Deadline on 25-Oct-2004
/RFDO LQVWLWXWLRQV GHDOLQJ ZLWK VSDPPLQJ
•
HKISPA “Anti-SPAM - Code of Practice” 2000
http://www.hkispa.org.hk/antispam/cop.html
Has Got
• ISP has default abuse account: abuse@ispname and should reply to complaints
• Include in AUP conditions of breach (e.g. spam) that could lead to termination of service
Has Not Got
• service level guarantee on response
• escalation path if no response
• authority to mandate all member ISPs to sign in
•
OFTA Guideline
•
Compliance by granting and removal of Branding
http://www.ofta.gov.hk/junk-email/index.html
Encourage industry to self-regulate
Educate user to avoid spam
HKCERT
Education and technical support, e.g. advise reporter to locate spamming IP address
No authority to ISPs
&URVV %RUGHU 6WUDWHJLHV
SC Leung
&URVV ERUGHU FROODERUDWLRQ
• Standardizing Legislation platform and enforce
Cross-border Jurisdiction on Spam
• Standardizing Incident Response
• Standardizing Spam Source/Content Information
Exchange
6WDQGDUGL]LQJ /HJLVODWLRQ 3ODWIRUP
• U.N.’s ITU WSIS Thematic Meeting on Countering Spam
(2004-Jul)
http://www.itu.int/osg/spu/spam/meeting7-9-04/index.html
Regulators from 60 countries + Council of Europe + WTO at meeting
U.N. plan to bring the international spam under control in 2 years by
standardizing legislation
Five-layered approach (by Robert Horton, Chairperson of the meeting)
•
•
•
•
•
Legislation
Technical measures
Industry Partnerships, esp. with ISPs, carriers and direct marketing associations
Education of consumers and industry players
International Cooperation at the levels of government, industry, consumer,
business and anti-spam groups
6WDQGDUGL]LQJ /HJLVODWLRQ 3ODWIRUP
•
Bilateral Agreements
Australia: MOU with Korea (Oct 2003), Thailand, UK and US (July 2004)
http://australianit.news.com.au/articles/0,7204,10042121%5e15306%5e%5enbv%5e,00.html
share info, exchange evidence and coordinate enforcement vs. cross-border spam violations
•
National Law
CAN-SPAM in USA (Jan 2004) not very successful
http://www.usatoday.com/tech/columnist/ericjsinrod/2004-03-25-sinrod_x.htm
Australia's spam laws successful (Jul 2004) but only works for 2% spam
http://australianit.news.com.au/articles/0,7204,10222221%5e16123%5e%5enbv%5e,00.html
Korea (Jan 2003)
http://www.mic.go.kr/eng/jsp/maj/maj_100_02.jsp?dept=1&m_code=p100_0064_1&curpage=
1
Reference for Spam Laws: http://www.spamlaws.com/
6WDQGDUGL]LQJ ,QFLGHQW UHVSRQVH
• No universal mechanism
CERT take care of local ISPs; coordinate with oversea CERTs
Relies on successful coordination with local ISPs to resolve
problems
7:
7:
6XSSOLHU
6XSSOLHU
/DZ
/DZ
(QIRUFH
(QIRUFH
,63
,63
6*
6*
$8
$8
,'
,'
&1
&1
7+
7+
Supplier
Supplier
-3
-3
$3&(57
$3&(57
0<
0<
.5
.5
,63
,63
+
+..
Law
Law
Enforce
Enforce
,63
,63
Supplier
Supplier
6WDQGDUGL]LQJ 6SDP
6RXUFH&RQWHQW ,QIR([FKDQJH
• No local blacklist nor local spam content database in
rising economies in Asia and Europe (HK has not got
them)
Be a good neighbour: we should tell our neighbours who are the
bad guys in our locality.
• Lack of data exchange standard of blacklist
/DZ
/DZ
(QIRUFH
(QIRUFH
/RRNLQJ IRUZDUG
• Safe Havens for Spam are widely available
• Scope of Spamming is expanding
SMS, Instant Messaging (SPIM)
• Un-moderated Mailing List is dying RSS Feed is rising
• Technical Aspect
Spammer and Counterparts are at war
Cocktail approach is most successful now
Proposals of fundamental changes in the SMTP infrastructure could alleviate
the problem but it takes longer time to realize
It is important to empower non-technical user to report spamming
• Legislation & Incident Handling Aspect
Getting global attention now

Presentation - Professional Information Security Association

Transcription

Similar documents

Parent Access to the - Collier County Public Schools

AppRiver guards your inbox against unwanted

HSW GmbH

Logging in to the System Please go to the following URL to begin

മൻഖൂസ് മൗലിദ്

Pushing the envelope—inside out

How to configure your email client

CHAMPVA Meds by Mail Brochure

What We`ve Seen in Q2 2015

Online Full Text

Opting Out of Spam: A Domain Level Do-Not

Fighting Spam For Dummies

A Personalized Arabic Spam Detection Model