Technion - Israel Institute of Technology

Technion - Israel Institute of Technology

© Technion - Israel Institute of Technology, Elyachar Central Library
‫ספריות הטכניון‬
The Technion Libraries
‫בית הספר ללימודי מוסמכים ע"ש ארווין וג'ואן ג'ייקובס‬
Irwin and Joan Jacobs Graduate School
©
All rights reserved to the author
This work, in whole or in part, may not be copied (in any media), printed,
translated, stored in a retrieval system, transmitted via the internet or
other electronic means, except for "fair use" of brief quotations for
. academic instruction, criticism, or research purposes only
.Commercial use of this material is completely prohibited
©
‫ת‬/‫כל הזכויות שמורות למחבר‬
‫ חיבור זה או‬,‫ להפיץ באינטרנט‬,‫ לאחסן במאגר מידע‬,‫ לתרגם‬,‫ להדפיס‬,)‫אין להעתיק (במדיה כלשהי‬
‫ ביקורת או‬,‫ הוראה‬,‫ למעט "שימוש הוגן" בקטעים קצרים מן החיבור למטרות לימוד‬,‫כל חלק ממנו‬
.‫ שימוש מסחרי בחומר הכלול בחיבור זה אסור בהחלט‬.‫מחקר‬
© Technion - Israel Institute of Technology, Elyachar Central Library
dncwd
.1
milcenn zixwird diitvd .milcen epnn mivnne (oeni` ze`nbec) rcin milawn micnel minzixebl`
.oeni`d ze`nbec z` ex`iz ea weic eze`a hrnk xewn eze`n sqep rcin ex`ziy :elilki mdy `id
ziaeyigd dcinld inegz ly cewinl xarne ,ze` ceair oebk mipeebn minegzn zeiraa daeyg ef dpekz
.mipelin zcinl `id mixweg ep` dze` efk zg` dira .dwihqihhqde
zen`ez opi`y ,zebixg ze`nbec ly ohw feg`l zepiqg `id micnel minzixebl`l dievx zxg` dpekz
dnvr zexyt`l s` zeribt lilkn mieqn milcen beqy dgkedl zelaewnd zehiyd ,xrvnl .xewn eze` z`
zilbp`ae ,(.n.n) "xcqen lewyn" mirvand minzixebl` ly zxbqn mirivn ep` .el`k mibixg meiw ly
,zeycge zeniiw dcinl zeira ly agx geeh lr dyaldl zpzip ef zxbqn .rcind ly
regularized weighting
zepekz mb migihand (ihnznd oaena) mihtyn mb zipnf eae ,zebixg ze`nbecl znieqn zepiqg dtiqen
xzei dxeva mze` xicbp ,`nbec jxc dlrnl mixkfpd zeirade migpend z` mibcp df rhw jynda .dllkd
.xeaigay ze`vezd xwir z` xewqp xivwzd x`yae ,zwiecn
mibevi zcinl :`nbec
1.1
.onf jxe`l zegwlpy zecicn qiqa lr zakxen zkxrna zelwz ly xezi`d z` oknl mivex ep`y gipp
e` ,EKG zecicn qiqa lr `l e` dpiwz `id al ly zilnygd zelirtd m`d hilgdl mivex ep` lynl
lk ,oeni` ze`nbec epl zepzip .zeipkn zecepz miccen ep` ea qgcn wecal i`pkhl `exwl m`d hilgdl
ep` .dfk dxwna lawi mzixebl`dy mivex epiid dze` dhlgdd z` mbe onf jxe`l dcicn zllek zg`
dcicn x`znd mixtqn xehwee ozpida :ix`pia dhlgd llk epiid ,beeqn on`l ick ze`nbeca miynzyn
itk beeqnl zecicnd z` xiarp m` .zeixyt` zeaeyz izyn zg` xifgi dhlgdd llk ,onf jxe`l zg`
dpan qtqtl lelr df ,illk dhlgd llkl zexiyi (zeipkn e` zeilnygd) zecehiltn`d xnelk ,erveay
.zerexb zehlgd lawl okle diraa aeyg
zqcpd `xwp ,beeqnd zlrtd iptl zecicnd lr lrtend ietinl enebxze ,dfk inipt dpan ly xezi`d
zervn`a mc` i"r rvean `ed xy`k ,(representation
engineering e` feature engineering zilbp`a) mibevi
zeni`znd zecehiltn`d ik xryl ieyr qcpdn lynl ;dirhe ieqipe daygn ,megzd lr rci ly aeliy
minzixebl`a yeniy `id dvetpe zxg` zexyt` .zecicnd ly zeaeyg zepekz od minieqn lb ikxe`l
.(representation
learning
zilbp`a) mibevi zcinl `xwi f`y ,jildzd z` oknl ick micnel
:ozervn`a df xeaig i`yep ody zeirad z` mibcpe ,mibevi zcinll zehiy yely x`zp dzr
-xewnd zecicnd ly cnindn jenpd
k
cninn ix`pil agxn zzl zecicnd z` lihp :cnin zcxed .1
gezip .dlhdd z`vez z` mix`znd mixtqnd
k
`id beeqnl xiarp eze` dcicn ly bevid .zei
zepeyd z` xnynd agxnd zz z` xgai ,lynl (Principal
Component Analysis)
miiy`x minxeb
.zixyt`d zilniqwnd
mix`znd mixtqn
k lawp jk ,el`l dcicn lk ly oeinc aygpe ,mireaw zeze` k xgap :oeinic zxnd
k .beeqnl hlwd jke ,bevid eidi el` .dcicnd z`
.odly ceair ze`vez e` ,(mixg`e mi`ixa EKG ly zehlwd) oeni`
.2
ze`nbec zeidl mileki mieeyn odil`y zeze`d
zix`pil divpianew `vnp ,D = (d1 , . . . , dk ) ,mireaw zeze` k -e x dcicn ozpda :lilc ceciw .3
Pk
Pk
i=1 αi di ≈ x
i=1 |αi | -y oaena e` ,qt`n mipey αi mincwndn hrn wxy oaena "dlilc" dpidy
mixcza qepiq milb hrn ly mekqk daizkl zeax minrt zepzip ,lynl ,zeipkn zecepz .ohw epid
xareny bevid deednα mincwnd xehwe .mihren miez ly mekq md xzpqt ililvy itk ,mipey
xei`a `nbec .beeqnl
i
© Technion - Israel Institute of Technology, Elyachar Central Library
x`zl ozip eze` ,mec`d ze`d i"r ahid miaxewn legkd ze`d ly mikxrd 128 .xg` qiqaa ,lilc ze`
.Haar beqn oelina cala mixai` 25 mekqk
ziaeyig dcinla zeira xtqn
exyt`iy lilc ceciw xear
D
1.2
zeze`d z`ivn ly dirad ef ;mipelin zcinl ly dirad z` dligza epxkfd
ccewnd oeliny ick zewitqnd ze`nbecd xtqn z` migzpn ep` .ze`nbecd ly xyt`d lkk wiecn bevi
,mipelin zcinl ly dnibcd zeikeaiq idef :xewn eze`n zeycg ze`nbec mb ahid ccewi ahid oze`
.dhnl wiecna xicbp dze` ,dllkd znkl dxev deednd
zixitn` dcin i"r onvra
P zenkeqne,dreci dpi`y ν zebltzd jezn zeribn zecicnd n ik migipn ep`
k
f` ,fD (x) = minα i=1 αi di − x :bevid z`iby ziivwpet z` fD : X → R+ onqp D oelin xear .µn
2
onqp ,zeillka .dkenp didz ,Ex∼ν fD (x) zeycg zecicn xear dietvd bevid z`iby oelin xegal epzxhn
zg` xgap okezn
xzeia ohwd
n
F = {fD }
xign zeivwpet zwlgn epl yi if` ,En f
`id dira ly dnibcd zeakxen if` .jenp didi
Ef
= Ex∼µn f (x)
-e
Ef = Ex∼ν f (x)
En f dxear
mb ik ze`xdl dvxpe ,jenp
0
:µn dycg dnibc lr minqg gikedl xyt`nd
Ef ≤ (1 + ω) En f + ε (n, ω, . . . )
z`iby `xwp
ε-e ω ≥ 0
xear ,dler
n-y
lkk 1-l zil`ivppetqw` dhepy zexazqda miniiwzn xy`
zewlgn migzpn ep` mipelin zcinl xear dnibc zeakxen zgkedl zeniiw zehiy lirtdl ick .dllkdd
.lilc bevi ly mipeyd mipaend z` zebviind
1000-l sqepa :mivex epiidyn xzei rext minrtl epid dcinl inzixebl` lirtdl dvxp eilr rcind
F
lirtnd mzxiviay zenibc 10 mb d`xpk eplaiw ,zelwelwe zepiwz zepekn ly zecepz ly zepiwz zecicn
`ven didy mzixebl` .dieak dziid dpekndy e` lwlewn did ccendy e` ,oekp `l ccend z` mwin
dxeva lrtei m` ekxrn daxd ca`l lelr oey`xd beqdn zecicnd 1000 z` xiaqdl wiecne heyt lcen
el` zrax`n cg` dfi`" wgyna enk .(y`xn mircei `l ep` ozedf z`y) zetqepd 10-d lr mb ziai`p
i` :x`yd z` x`znd lcen z`ivnn cxtpa mibixgd z` zedfl illk ote`a ixyt` df oi` ,"llkdn `vei
,epxrvl .zelibxl zebixg zecicn oia dpigand dcigid ziaihwiiae`d dpekzd zeidl dlelr lcenl dn`zd
xign zelra zecicn ,d`ibyd reaix lynl ,zeax xign zeivwpet zgz ;zeaiyg dze` dcicn lkl dvwn
En
miililc mibevil oelin e` cnin zcxed cenll dyw okl .`vnpy lcend lr mb xzei zerityn xzei deab
dzegt zeaiyg wlgly jk zecicnl milwyn zzl `id dvetp dwipkh .mibixg zellek ze`nbecd xy`k
miyxec onvr ze`nbeca miielzd milewyn .(qt` lwyn ozn i"r zecewpn mlrzdl ozip s`e) mixg`ln
oebk xzei miillk minqg lelkl dnibc zeakxen gpend ly dllkd
Eµ0n L (µ0n , f ) ≤ (1 + ω) L (µn , f ) + ε (n, ω, . . . )
`weec `le)
µn
-a mipezpd lr
f
ikxr ly zillk divwpet `ede
L (µn , f )-a
oneqn llekd xignd my
ep` eze` xbz` if` .dcinld zxez ly minieqn miihxcphq milka yeniy rpen df iepiy .(mdly rvennd
,witqn `ed zei`xdn lecb feg` oda zeiral icnl zeillk jexry zeira zxnd `ed df xeaiga mitwez
milcen xtqn `l` cg` lcen `l cenll mipiipern ep` xy`k dxingn ef dira .dyibd z` zeabl dixe`ze
mibivn ep` .dhnl xei`a enk ,eli`l zekiiy ze`nbec eli` lr oezp rcin `ll ,cg` ze`nbec xb`n jezn
zeniiw eidy dllkdd ze`vezl zeizernyn zeagxd milawn mixwndn cg` lkae ,mixwnd ipyl zehiy
.zenec zeiral mcew
ii
© Technion - Israel Institute of Technology, Elyachar Central Library
Our association method
3
2
Lloyds based association method
4
Data
Robust joint loss 0.20 correct on 38 points
Robust joint loss 0.20 correct on 40 points
Data
Minimum loss 0.20 correct on 14 points
Minimum loss 0.20 correct on 22 points
3
2
1
1
0
0
−1
−1
−2
−2
−3
0.0
gxkdd
0.2
0.4
0.6
0.8
−3
0.0
1.0
0.2
0.4
0.6
0.8
1.0
.dly xzeia aehd lcend z` dcewp lkl zqgiind dvetp dhiy ze`vez mi`ex ep` oinin
ly ohw xtqn xiaqdl zeleaxtdn zg` dhn ,d`ibya ireaix xign aeliya ,zecewpd lkl qgiizdl
dze`a dlrnl oini cva zeidl zeleki zebltzd dze`n zetqep zecewp :dllkd xqeg deedn mb df .mibixg
ohw wlgn minlrzn (l`ny cva) eply dhiya .exgapy milcend zgz ddeab d`iby lawle ,zexazqd
.oaexl ce`n aexw mixaere zecewpdn
df xeaiga zeixwir zenexz
.McDiarmid oeeiy i`l ycg yeniy i"r mekq opi`y zeivwpetl dllkd zgkedl zehiy ly dagxd .1
xy`n zeaeh dllkd ze`vez mini`zn mikxryn xear gikedl xyt`nd df beqn ycg oeieeiy i` .2
.mixignd rvenn xear zepinf
miavw mr llek ,mincwn ly ohw xtqn mr lilc bevil miynynd mipelin xear dllkd ze`vez .3
.mixidn
:ze`ad zepekzd dl xcqen lewyn zinzixebl` zxbqn .4
`qtewk zrvend zxbqna dze` sehrl ozip ,mixign ly rvenn znvnvnd jexry zira ozpda (`)
.dxegy
.zelwa zecewp ipeiln cr ,icnl zelecb zeiral dlrtd zxa (a)
:mini`zn mi`pz zgz (b)
dxeva miielz mewnae ,zinzixbel dxeva wx zilniqwnd d`ibya miielzd dllkd inqg .i
.lcenl ziqetihd d`ibya wx dlibxd
.oekp rawp xhnxtd xy`k zinewn zix`pil zeqpkzd .ii
:mini`zn mi`pz zgz xcqen lewyn ,miaexn milcen xear .5
.cgi milcend lk iabl zeniiw ze`vezn licadl ,cxtpa lcen lkl dllkd ze`vez zwtqn (`)
-ewp itl` ze`na zeira oexzt mixyt`nd milwyn znydl zeihwxt zehiye zeikeaiq inqg (a)
.milcen ze`ne zec
iii
.2