MIMIC: Finding Optima by Estimating Probability Densities Jeremy S

MIMIC: Finding Optima by Estimating Probability Densities Jeremy S

#%$'&()!*+, - ./ ++./
!"
021+3416587(9;:=<>1@? ACB;16DFE=GH;I34JK1MLN,:;OPLRQ=16JSJTEU0236:VEXWIY;J=Z>[SACJKI
\]_^T`ba2cd`Ve4fUgih6^Tjkflfm`lnojkhpcdj qUesrpto]ues^_tF]Tv
w esx_xdeFcTy6zx_jk^T^_xgihxT^_`l^_z^Tj t4{;|;jPcTyhtFfltonFv
},e4~r]_`V€6nFjo w \‚oƒ„†…F‡
ˆŠ‰Œ‹oŽ;C
gih‘~’esh+vto“^_`l~>`m”†e4^_`ltoh“]_torfljk~•x†^_yj x_^T]_zpcd^Tz]_jts{=x_tFflz^T`ltFhx,]Tjd–pjPcd^Tx
cdtF~>“fljd—•]_jkfVes^_`ltFhxTy`l“x˜rpjk^i™˜jkjkhš^_yj €6`œ›2jk]_jkh6^`mh“z^˜“pe4]de4~•jR^_jk]TxP=žtF]
jd—pe4~•“fmjo4jd—“pjR]_`ljkhpcdj~’e†v^_jkflfzxŸ^_ypes^ c¡jk]_^de4`lh“pe4]de4~>jk^_jk]Tx¢e4]TjŒcdfltFxTjkflv
]_jkfVe4^TjP€(eshp€8x_ytFzfV€8hto^ rpjj¡—M“fmto]_jP€£`lhp€6jk“Cjkhp€6jRh+^Tflv+‘¤6`l~>`mfVes]_flvFj¡—+¥
“pjk]T`ljkhpcdj¦~’e†v>jkx_^de4rfl`lx_y!^Type4^'e•xTzrxTjk^ts{§“Ce4]ues~•jk^Tjk]_x~zx_^^ue4¨ojtFh
“pe4]T^_`VcdzfVe4]©FesflzjkxPª\Œh6v!xTjPe4]dc_y«t4{¬^Tyj‘cdtFxT^•fVe4hC€6xuckes“pj¦xTytFzfV€£^ue4¨oj
eF€6©Fesh+^de4noj,t4{U^_yjkx_j]_jRf­es^_`ltFhx_y`l“x†U®!j“]_jkxTjkh6^ w g w g_}6e{S]de4~•jR™¢to]_¨
`mh¦™,y`VcTy•™˜j e4hpesflv+”Rj¬^Tyj nofmtorpe4fCx_^T]_zpc¡^_z]Tjts{X^_yj¬to“^_`l~>`m”†e4^_`ltohfVe4hC€o¥
xuckes“pjF£\¯ht4©ojkf,e4hp€8jd°’c¡`mjRh+^e4flnotF]_`l^Ty~±{Sto] ^_yj’jRx_^_`l~¦e4^_`ltoh‘t4{Œ^Ty`lx
x_^_]Tzpcd^Tz]_j `lx,€6jk]T`m©ojP€2;®!j,zxTj¨+ht4™,fmj†€6nFj,ts{U^_y`lx˜x_^_]Tzpcd^Tz]_j ^Tt nFz`­€Mj¬e
]ue4hC€6tF~>`m”RjP€>x_jPes]ucTy@^Ty]_toznFyš^_yj'x_tofmz^_`ltFh¦x_“CeFcdje4hp€²p`lh¦^_z]_hUC^_t>]_jd¥
aphj˜tFz]³jkx_^T`m~¦e4^Tj=t4{^_yj¢xT^_]_zCcd^_z]TjF¢´z]³^_jPcTyh`­µozj˜tFr^ue4`lhx³x_`lnFh`ba2cke4h6^
x_“CjkjP€šn+e4`lhx˜t4©FjR],tF^_yjk]Œ]de4hp€6to~•`l”kjP€'to“^_`l~>`m”†e4^_`ltoh“]_tcdjP€Mz]_jkx†
¶
·+¸ MŽ¹,ºŒ»ŒC¼T¹ ¸
½ `l©FjRh‘x_to~•j cdtFxT^{Szhpcd^T`ltFh!¾'¿ÁÀCÂ,™,`l^_yšfmtckesfU~>`mh`m~¦e66™¢j~’e†v•xTjPe4]dcTy‘{Sto]^_yj'tF“^T`l~’e4f
ÀŠ`lhŠ~’esh+v!™,e†v6xPÃ¢es]_`Ve4^T`mtohx't4{ nF]deF€6`ljkh6^•€6jkxdcdjkh6^@es]_j‘“Cjk]_yCe4“x>^_yj‘~>tFxT^“pto“zfVe4]†
®ÄyjRh@~>tFx_^t4{,^_yj~>`lh`l~•ees]_j{Áe4]{S]TtF~±tF“^_`l~’esfÁ^_yjxTjPe4]dcTy!~zx_^jk`l^_yjk]`mhCcdflzp€6je
r]Tz^_jd¥K{Sto]ucdj‘cdto~•“CtFhjkh6^ to]`lhpc¡tF]_“CtF]de4^Tj•]ueshp€6to~•`l”Pe4^T`ltFhU‘}˜fVe4xTx_`Vckesf¢jd—pes~•“fljkx `lhpc¡fmzC€6j
¤6`l~zfVe4^TjP€\hhjPesfm`lhnÅ¿Á¤6\Â=e4hp€½Æjkhjk^T`­cÇ\ŒflnFto]_`l^_y~•x˜¿Á½\xu˜¿iÈ`m]T¨6“pe4^T]_`VcT¨É6½jkfVes^_^¢eshp€
Ã=jPcRc_y`Á;„†‡FÊF…ËÌ,tofmfVeshp€22„P‡oÍFÎoÂk=gih‘esfmfUcRe4x_jRxP2`lh¦^_yj“]_tcdjkxTxt4{=tF“^T`l~•`l”k`lhn¾'¿iÀp˜~¦e4h6v
^_ytFzxde4hp€6x˜tF]Ÿ“pjk]Type4“x,~•`lflfm`ltohx2t4{Uxde4~•“fmjRx=t4{X¾'¿iÀpŸe4]_jjk©Fe4flzpes^_jP€² w tox_^˜tF“^T`l~•`l”Pe4^T`ltFh
e4flnotF]_`l^Ty~•x ^de4¨oj’^TyjkxTj‘~•`lflfl`mt4hx t4{“`ljPcdjkx>t4{ `lh6{StF]T~’es^_`ltFhU;e4hC€(cdto~•“]Tjkx_x>^_yjk~Ï`lh6^_t
e¦xT`mhnFflj“CtF`lh6^ÀMÐÄ^_yj•cdz]_]_jRh+^'jkxT^_`l~’es^_jt4{,^_yj•x_toflz^_`ltFhª¿Stohjhto^ue4rfmjj¡—c¡jk“^_`ltohÑe4]Tj
½\x^Tt!™,y`Vc_yª™¢j¦™,`lflf¢]Tjk^_z]_hŠx_yto]_^TfmvÂR(gi~¦e4nF`lhj'x_“fm`l^T^_`lhn8^_yj¦x_j†e4]ucTyŠ“]TtMc¡jkx_x¦`lh6^_t
^i™˜t“pes]_^TxP6rCtF^_y>^ues¨+`lhnÆÒuÓFƒ^T`m~>j,x_^Tjk“xP˜Ô=to^_y¦“pes]_^_x,e4]Tjx_^_]Tzpcd^Tz]uesfmflv`V€6jkh6^T`­cRe4fiÕU^de4¨6`lhn
e(€Mjkxucd]T`l“^_`ltFht4{¾'¿ÁÂR,^_yjRvÄx_^de4]_^¦^Tyjk`l]•xTjPe4]dc_y{K]_tF~xTtF~>j’`lh`l^T`­esf,“pto`lh+^†|,yj@xTtFflj
rCjkhjdap^jRhPÖdt4vFj†€!r6v!^_yj>x_jPc¡tFhp€ª“pes]_^'t4{Œ^_yj’xTjPe4]dcTyÑt4©ojk]'^_yjÅap]_xT^`mx'^_yCe4^ ^Tyj¦`mh`m^T`Ve4f
“CtF`lh6^,`mx˜“Cjk]_ypes“x cdfltFxTjk],^_t^Tyj tF“^_`l~z~‘2gih6^_z`m^T`l©Fjkflv6F^Tyjk]_j~zx_^˜rpjx_tF~>j¬eo€€6`l^_`ltFhCe4f
`lh6{Sto]_~’es^_`ltFh^Type4^Œc¡tFzfV€¦rCj fljPe4]ThjP€>{K]_tF~ ^_yjap]_xT^Œypesfœ{ ts{=^_yj xTjPe4]dcTyUC`b{=tohflv•^_t™,e4]Th
^_yj¬xTjPcdtohp€¦ypesfb{=e4rCtFz^Œe†©otF`V€esrflj,~•`lx_^de4¨FjRx§eshp€>“`l^i{Áesfmflx†
®!j“]Tjkx_jkh6^•esh!tF“^T`m~>`l”Pe4^T`mtoh@e4flnotF]_`l^Ty~ ckesflfmj†€ w z^_zCe4fœ¥ÁgihM{StF]T~’e4^T`ltFh6¥ w eR—`l~`l”k`lhngih6¥
“z^ }˜flzxT^_jk]T`mhn@¿ w g w g_},ÂRpgi^ es^_^_jR~•“^Tx¬^Tt’cdto~•~zh`­cRe4^_j`lh6{StF]T~’es^_`ltFh¦e4rCtFz^^_yj cdtox_^
{SzhCcd^_`ltFhštFr^de4`lhjP€>{K]_tF~ tFhj `l^_jk]de4^_`ltoh‘t4{=^_yjxTjPe4]dc_y8^_t>fVe4^_jR]¬`l^Tjk]ues^_`ltFhxt4{=^_yjxTjPe4]dc_y
€6`l]_j†cd^_flv6Œgi^ €6t6jkx^_y`mx`lh@e4h‘j¡°’cd`ljkh6^Æeshp€¦“]T`lhpcd`l“fljP€š™§e†v6,|,yjk]Tj es]_j'^i™¢t>~’es`mh¦cdtF~Å¥
“CtFhjkh6^Tx t4{ w g w g_}Õap]_xT^P;e•]de4hp€MtF~•`l”kj†€•to“^_`l~>`m”†e4^_`ltoh esflnFtF]T`l^_y~ ^_ypes^xue4~>“fljkx,{K]_tF~
^_ytFx_j,]TjknF`ltohx;t4{p^Tyj¢`lh“z^;x_“peocdj,~•tox_^Ufl`l¨FjRfmv^_t cdtoh+^de4`lh ^Tyj,~•`lh`l~'z~Š{Sto]X¾'¿iÂkË4xTjPcdtFhC€2
e4hšjd›Éj†cd^_`l©Fj€6jkhx_`l^iv‘jkxT^_`l~’es^_tF]˜^_yCe4^ cke4hšrpj zx_jP€š^_t¦cke4“^Tz]_je™,`V€6j©Fe4]_`ljk^iv>t4{¢xT^_]TzpcT¥
^_z]_jtFh8^_yj`lh“z^¬xT“peFc¡jFUvojk^`lxÆc¡tF~•“z^uesrflj{S]_to~±x_`l~•“fmjx_jPcdtohp€8tF]d€6jk]x_^de4^T`mxT^_`Vcdx toh
^_yjƀe4^ue w g w g_} x˜]_jkxTzfl^_x,tFh¦xT`m~>“fljŒcdtFxT^˜{Szhpc¡^_`ltFhx˜`lhp€6`Vckes^_j¬esh¦tF]u€Mjk],t4{;~’e4noh`l^_zp€Mj
`l~•“]Tt4©Fjk~>jkh6^Œ`lh(“Cjk] {StF]T~’e4hCcdj•t4©ojk]]TjkfVe4^TjP€Še4““]Tt+eoc_yjkxP(žz]T^_yjk]j¡—M“Cjk]T`m~>jkh6^_x'tFhŠe
¨o¥ c¡tFfltF]˜~’es“’cdtofmto]_`lhn'“]_torfljk~ v6`ljkfV€xT`m~>`lf­es]=`l~•“]Tt4©Fjk~>jkh6^_x†
³ º
Ϲ=Ž
w esh+vš™¢jkflf;¨6ht4™,h8tF“^T`m~>`l”Pe4^T`mtoh•“]TtcdjP€6z]Tjkxhjk`l^Tyjk] ]Tjk“]_jRx_jkh6^'htF]z^T`mfl`l”kj'^_yjxT^_]TzpcT¥
^_z]_jt4{;^_yj to“^_`l~>`m”†e4^_`ltoh fVe4hp€6xdcke4“CjFŸgih’cdtoh+^T]uesx_^PC½jkhjR^_`Vc\ŒflnotF]_`l^Ty~•x˜¿Á½Æ\¬Â,e4^T^_jk~>“^
^_t ckes“^_z]_jŒ^Ty`lx=x_^T]_zpc¡^_z]Tj¬r6vesh•eF€ytc˜jk~rCjP€€6`lhnt4{2^_yj,“pe4]de4~•jR^_jk]Tx=tFh6^_t efl`lhjŒ¿K^_yj
cTy]_to~•tox_tF~>jPÂk²|,yjŒ`lh6^_jkh6^˜t4{U^Tyj¬cd]TtFxTx_t4©FjR],tF“Cjk]ues^_`ltFh>`mhxT^ueshp€e4]d€>nFjkhjR^_`VcŒesfmnotF]T`m^Ty~•x
`lx¢^Tt“]_jRx_jk]T©Fjeshp€>“]_to“pe4n6e4^_je no]_toz“¦t4{X“Ce4]ues~•jk^Tjk]_x˜^_yCe4^2rpj“pes]_^_`Vesfmflv']_jkxT“ptoh6¥
x_`lrflj{KtF],nFjRhjk]ues^_`lhn¦e'{Áe†©otF]uesrfljjk©Fe4flzpes^_`ltFhU;©FjRh‘™,yjkh8x_zpcTy‘no]_toz“xjd—`lx_^PC~¦e4h6vt4{
^_yjt4›2x_“]T`mhn’nojkhjk]de4^TjP€!€6t¦htF^“]_jkxTjk]_©oj•^Tyjx_^T]_zpcd^Tz]_j>t4{¢^TyjkxTj•nF]TtFz“x¬rCjPckeszx_j>^_yj
cTytF`Vcdjts{Ÿc¡]_tFxTx_t4©ojk]Œ“CtF`lh6^¢`lx,]de4hp€6to~@
gihē]TtFrfljk~>x™,yjk]Tj‘^_yj8rCjkhjdap^¦ts{e8“Ce4]ues~•jk^Tjk]•`lx¦cdtF~>“fljk^_jkflv!`lhp€6jR“pjkhC€6jkh6^¦t4{ ^_yj
©Fe4flzj,ts{Xe4flfto^_yjk]˜“pes]ue4~>jk^_jR]_xPo^_yjŒ“CtF“zfVes^_`ltFhxT`m~>“flvjkhpcdt€6jkx,`lh6{KtF]_~¦e4^T`mtohe4rCtFz^=^_yj
“]TtFrpesr`lfm`l^iv•€6`lxT^_]_`lrz^_`ltFh‘t4©ojk],jPeoc_y8“pes]ue4~>jk^_jR]P˜gih‘^_y`mxcke4xTjF2^Tyjcd]TtFx_xTt4©Fjk]to“pjk]de4^T`mtoh
`lx¢j†µFz`m©Fesfljkh6^¢^Ttxue4~>“fl`lhn{S]_to~ ^_y`lx,€6`lx_^T]_`lrz^T`mtohUË6^_yj¬~>tF]Tj¬cd]TtFx_xTt4©Fjk]TxŒ^Tyjrpjk^T^_jk]^_yj
xues~•“fljF;©Fjkh>`lh“]_torfljk~•x;™,yjk]Tj,ap^ThjkxTx,`mxŸtFr^de4`lhjP€'^Ty]_toznFy^Tyj¬cdto~r`lhjP€jd›2jPcd^_xt4{
cdflzxT^_jk]TxŒt4{;`lh“z^TxP^Tyj ½\"cd]_tox_x_t4©ojk]Œto“pjR]ue4^T`ltFh¦`lx¢rCjkhjda2cd`Ve4fUtohflv™,yjkh¦`l^_x,]de4hp€6to~•flv
cTytFxTjkh(cdflzxT^_jk]Txypes““Cjkh!^_t‘cdfltFxTjkflv@~¦e4^dc_yš^_yjzhC€6jk]_flv6`lhn¦x_^T]_zpcd^Tz]_j>t4{,^_yj“]_tFrfmjR~@
ԟjPcke4zx_j•ts{§^Tyj]ues]_`l^iv‘ts{¢x_zpcTy(eÅ{Sto]_^_z`m^TtFzxtckcdz]_]_jRhpcdjF˜^_yjrpjkhjdap^ ts{§^Tyj•cd]_tox_xTt4©Fjk]
tF“Cjk]de4^_`ltohÑ`lx>nF]_j†e4^_flvŠ€6`l~•`lh`lx_yj†€2Ä\x’e4x]Tjkx_zfm^†¬½Æ\Œxype†©oj@e!cTyjPcT¨Fjk]TjP€y`lx_^TtF]Tv(`lh
{SzhCcd^_`ltFhštF“^T`m~>`l”Pe4^T`mtoh‘¿iÔ¢esz~@2ԟtFhjky!e4hC€@½ es]_]Tjk^_^†=„P‡o‡FÎË2q³e4hnC=„P‡o‡FÎFÂR¬´hjts{¢tFz]
nFt6e4flx˜`mx,^Tt`lhpc¡tF]_“CtF]de4^Tj `lhx_`lnFy6^Tx={S]TtF~±½\Œx,`lh¦e'“]_`lhpcd`l“fljP€ÅtF“^T`l~•`l”Pe4^T`ltFh'{S]de4~•jR™¢to]_¨2
|,yjk]Tj‘ype†©oj•rCjkjkhÄto^_yjk]•es^_^Tjk~•“^Tx^_t!cke4“^_z]Tj‘^_yj‘eo€6©Fe4h6^uesnFjkxts{ƽ\xP³tF“zfVes^_`ltFh
Ԙe4x_j†€•gihCcd]_jk~>jkh6^uesfUq2jPes]_h`lhn>¿=ԟg_qUÂ,es^_^Tjk~•“^Tx=^_t'`lhpcdto]_“CtF]ues^_j^_yj¬htF^_`ltoht4{Xe'cke4hp€M`b¥
€es^_j “CtF“zf­es^_`ltFh>r6v’]Tjk“fVeFc¡`mhn`m^,™,`l^Ty eÅx_`lhnofmj“]TtFrpesr`lfl`m^iv©ojPcd^TtF] ¿iÔ¢esflzPÖkee4hC€’},es]_z6¥
e4hCe2„P‡o‡FÎoÂk=eoc_yšjkfljk~•jkh6^,ts{X^_yj ©FjPc¡^_tF]`lx,^_yj “]_tFrCe4r`lfl`l^iv^Type4^e'“pe4]T^_`VcdzfVe4],r`m^˜`lh‘e
x_toflz^_`ltFh¦`lxŒtohUŒz]_`lhn^_yjfljPes]_h`lhn“]Ttcdjkx_x†U^Tyj'“]_torpe4r`lfl`l^iv©FjPc¡^_tF] ckesh‘rCj^TytFznoy6^
t4{Œe4x e>x_`l~>“flj ~•t€6jkf;ts{§^TyjtF“^T`l~•`l”Pe4^T`ltFh•fVeshp€6xucRe4“CjF•ÔŸ`m^Tx ™,ytox_j©Fe4flzjRxÆes]_j'ap]T~•flv
jkxT^ue4rfm`lxTyjP€yCe†©FjŒ“]_tFrCe4r`lfl`l^_`ljkx;^_ypes^§es]_jcdfltFx_j^_t•„ÇtF],‚³|,ytFxTjŒ^_yCe4^,e4]_jx_^T`mflfzh¨6ht4™,h
ype†©oj¬“]_tFrCe4r`lfl`l^_`ljkx,cdfltFx_j^Tt•‚ Ν
®ÄyjRh`l^;`lxX^Tyj"!$#%&!'(2ts{p^_yjŒcdtF~>“ptohjkh6^_x;t4{2e¬cRe4hp€6`V€es^_j˜]ues^_yjk]Ÿ^_ypesh^Tyj,“pe4]T^_`VcdzfVes]
©Fe4flzjRxŒt4{˜^_yjcdtF~>“ptohjkh6^_x^_ypes^¬€6jR^_jk]T~•`lhjkxyt4™"`l^,{ie4]_jRxPC`l^¬ckesh@rCj €M`b°’c¡zfl^,^_t¦~•t4©oj
=ԟg_q x]_jk“]_jkxTjkh6^ue4^T`ltFh!^_t4™,e4]d€6x¬eÅ©6`­esrflj x_toflz^_`ltFh³*)Çjk©Fjk]T^_yjRfmjRx_xP³jk©ojkh@`lh‘^TyjkxTjx_tF]T^_x
t4{=“]TtFrfljk~>x+=ԟg_q(ts{S^_jRh‘tFz^ ¥Á“Cjk] {StF]T~•x,nFjRhjk^_`Vc'e4flnFto]_`l^_y~•x˜rpjPcRe4zxTj^_ytox_jesfmnotF]T`m^Ty~•x
e4]Tjy`lhp€6jk]TjP€¦r6v•^Tyj{ÁeFcd^,^Type4^,]de4hp€MtF~¯cd]TtFxTx_t4©FjR]_x e4]Tj¬`lh6{K]_jPµozjkh6^Tfmv>rpjRhjdaÉcd`VesfÁ
\±©FjR]_vрM`mxT^_`lhpcd^†˜rz^]_jRf­es^_jP€ª^TjPcTyh`Vµozj‘™,e4x“]TtF“CtFx_j†€Šr6v(¤e4rCjkx’eshp€-,Fto]u€esh({KtF]e
]_jR`mhM{StF]dcdjk~•jRh+^>fljPe4]Th`lhn!^uesx_¨¿Á¤esrpjRx’e4hC€,Fto]u€eshU „P‡o‡FÎoÂkÄgih«^_yjk`l]Å{S]ues~•jk™˜tF]T¨ÉŸ^_yj
fljPe4]Thjk] ~zx_^Œnojkhjk]de4^Tj•eFcd^T`mtohx xTt’^Type4^ e>]_jR`mhM{StF]dcdjk~•jRh+^{Szhpcd^_`ltoh/ckesh@rCj•cdto~•“fljk^Tjkflv
jd—“fltF]TjP€2Š¤6`l~zfl^ueshjktFzx_flv+³^_yj’fljPes]_hjR]~zx_^'jd—“fltF`l^'™,ype4^'`l^ypesxfljPe4]ThjP€ªx_t!e4x^Tt
tF“^_`l~•`l”kj^_yj fltohn4¥i^_jk]T~ ]_jR™§es]u€2,¤esrpjkxe4hp€ ,otF]u€e4h@cTytox_j ^Tt•cdtFhxT^_]Tzpcd^ eÔ=tofl^_”k~¦e4hh
€6`lx_^T]_`lrz^T`ltFhš{S]_to~ ^_yj]_jk`lh6{Sto]ucdjR~•jkh6^{Szhpc¡^_`ltFhUÕ ¿ÁÀC ™,yjk]_j ¿iÀpÂ`lx^_yj
]_jR`mhM{StF]dcdjk~•jRh+^²{Szhpc¡^_`ltFh{Sto]UeFcd^T`mtoh/@`lxU^Tyj=^Tjk~•“Cjk]ues^_z]TjFFeshp€’`lxXe˜htF]_~¦e4fl`l”Pe4^T`ltFh
{Áeocd^_to]P'|,yjkv‘zx_j^_y`mx €6`lxT^_]_`lrz^_`ltFh‘^Tt¦nFjkhjR]ue4^Tj’eocd^_`ltFhxP \,^y`lnoy@^Tjk~•“Cjk]ues^_z]Tjkx ^Ty`lx
€6`lx_^T]_`lrz^T`ltFhe4““]Tt+eoc_yjkx=^_yj,zh`œ{StF]T~€6`lx_^T]_`lrz^T`mtohUFeshp€ ]Tjkx_zfm^Tx=`lh]de4hp€MtF~j¡—M“fmto]ues^_`ltFh
t4{¿ÁÂR\,^fmt4™)^_jk~>“pjR]ue4^Tz]_jRx tFhflv‘^TytFxTj•eFcd^T`ltFhx ™,y`­cTy8n+es]_hjk]f­es]_noj]Tjk`lh6{StF]dcdjk~>jkh6^
e4]Tj nFjRhjk]ues^_jP€²¬ÔŸv¦]_jP€6zCcd`lhn ^_yj fljPes]_hjR]¬“]_tFno]_jkxTx_jkxÆ{S]_to~¯e4h‘`lh`l^T`­esflfmv']ueshp€6to~•`l”kjP€
x_j†e4]ucTy¦^_t•e~•to]_jŒ€6`l]TjPcd^_j†€¦x_jPes]ucTy‘e4rCtFz^˜^_yj¬^T]_zjto“^_`l~’esfpeFc¡^_`ltFhU;gih6^Tjk]_jkxT^_`lhnofmv66^_yjk`l]
jkxT^_`l~’es^_j,{KtF] ¿iÀp˜`lx˜^_txTtF~•jjd—^_jkh6^e'~•t€6jkfCt4{;^_yj to“^_`l~>`m”†e4^_`ltohfVe4hC€6xuckes“pj™,y`VcTy¦`lx
cdtohx_^T]_zpcd^TjP€‘€6z]_`lhn ^TyjŒfljPes]_h`lhn “]_tcdjkxTxP˜|;t tFz]Ÿ¨+ht4™,fmj†€6nFjF6¤esrpjkxŒeshp€,Fto]u€esh•ype†©oj
hjk`l^Tyjk]¬es^_^Tjk~•“^TjP€>tF“^T`l~•`l”Pe4^T`ltFht4©FjR],y`lnFy’€M`m~>jkhxT`mtohpe4fCx_“CeFcdjkx†2hto]Œe4^_^Tjk~•“^_jP€>^_t'aC^
¿iÀp˜™,`l^Ty’ecdto~•“fljd—~•t€6jkfi
! ·"! ·$#
Èht4™,`lhn‘hto^_y`lhnšjkflx_j’esrptoz^¾'¿ÁÀCÂ`l^~>`lnFy6^Œhto^rCj•zh]_jPesx_tohpe4rflj>^_t8x_jPes]ucTyª{StF] `l^Tx
~•`lh`l~'z~ r+v8nFjRhjk]ues^_`lhn8“pto`lh+^Tx{S]TtF~
e¦zh`œ{Sto]_~ €6`lx_^_]T`lrz^_`ltoh!t4©Fjk] ^Tyj•`lh“z^_x ¿iÀpÂR
¤6zpcTyex_jPes]ucTy esflfmt4™,x³htFhj¢ts{p^_yj˜`lh6{Sto]_~’es^_`ltFhnFjkhjk]ues^_jP€'r6v “]Tjk©6`ltFzx;xues~•“fljkx³^_tjd›Éj†cd^
^_yjnFjkhjR]ue4^T`ltFh8t4{¢xTzrx_j†µFzjkh6^xde4~•“fmjRxP )ÇtF^x_z]_“]T`mxT`lhnFflv6p~zCc_yšfmjRx_x ™˜tF]T¨’~>`lnFy6^,rpj
hjPc¡jkx_xde4]_vª`œ{Œxue4~>“fljkx™¢jR]_j•nojkhjk]de4^TjP€£{S]_to~ eš€6`lx_^_]T`lrz^_`ltohU &% ¿ÁÀCÂk;^Type4^`mx'zh`œ{Sto]_~>fmv
€6`lx_^T]_`lrz^TjP€t4©ojk]=^TytFxTj¬À xŸ™,yjk]_j ¾'¿iÀpÂ('*) e4hC€'ype4x¢eƓ]_torpe4r`lfl`l^iv t4{X‚Æjkflx_jk™,yjR]_jF˜žto]
jd—pe4~>“fljF6`œ{X™˜jypeF€¦eFcRcdjkx_x^_t +%-, ¿ÁÀC˜{Sto].)0/1Ä~>`lh32¢¾'¿iÀpÂ,e'xT`lhnFfljxues~•“fljŒ™˜tFzf­€Årpj
x_zM°’cd`ljkh6^,^_t'aphp€¦esh’to“^_`l~z~‘
|,y`lx `lhxT`mnoy6^¬xTznFnojkx_^Tx e>“]Ttcdjkx_xts{§xTzpckcdjRx_x_`l©oj es““]Tt†—M`l~¦e4^_`ltohUÕ,nF`l©ojkh(e¦cdtFflfljPcd^T`mtoh!t4{
“CtF`lh6^_xŸ{StF]=™,y`­cTy¦¾'¿ÁÀCÂ('4)65e €6jkhxT`l^ivjkx_^T`m~¦e4^TtF];{Sto] &%87 ¿iÀp `mx,cdtohx_^T]_zpcd^TjP€2,ž]_tF~^Ty`lx
€6jkhx_`l^iv!jkx_^T`l~’e4^TtF]eo€€6`l^_`ltohpe4f=xde4~>“fljkx es]_j>nFjkhjR]ue4^TjP€2,ešhjk™ ^Ty]_jkxTytFfV€ªjkxT^ue4rfm`lxTyjP€2
)9.*)65(:<;Peshp€•ehjk™€6jkhxT`m^ivjRx_^_`l~¦e4^_to]§c¡]_jPes^_jP€2Ÿ|,yjŒ“]_tcdjkxTxŒ`lx¢]Tjk“CjPe4^TjP€¦zh6^_`lfC^_yj
©Fe4flzjRx,t4{ ¾ ¿ÁÀC¢cdj†e4x_j'^_t'`l~•“]Tt4©FjF
|,yj w g w g_}¯esflnFtF]T`l^_y~ rCjknF`lhxr6v(nojkhjk]de4^T`mhnÑeš]ue4hC€6tF~ϓpto“zfVe4^T`ltFh(ts{Æckeshp€6`V€es^_jkx
cTyt6tFxTjkh@zh`œ{StF]T~•flv'{S]_to~±^_yj'`lh“z^x_“peocdjF ž]_to~±^_y`lx“pto“zfVe4^T`ltFh¦^_yj'~•j†€6`Ve4h>ap^_hjkx_x
`lx,jd—^_]deFcd^TjP€‘e4hC€•`lxŒ€6jRhtF^TjP€)=56;|,yj esfmnotF]T`m^Ty~^_yjRh‘“]_tcdjRjP€6xPÕ
„o?>,“2€e4^Tj ^_yj“pe4]de4~•jR^_jk]Tx¢ts{ ^Tyj €6jkhxT`l^iv•jkxT^_`l~’es^_tF]˜t4{ &%@ ¿ÁÀCŸ{S]_to~¯e'xues~•“fljF
ƒ,½jkhjk]de4^Tj ~•to]_jŒxde4~>“fljkx={K]_tF~ ^_yj€M`mxT^_]T`mrz^_`ltFh +%A@ ¿ÁÀCÂk
…,¤6jk^B)CED 9 j†µFzCe4f˜^_t‘^Tyj )Ç^_y!“Cjk]ucdjRh+^T`lfmj>t4{Œ^Tyj•€e4^deGFÇjk^ues`mh8tFhfmvš^_yj>“pto`mh6^Tx
fmjRx_xŒ^Type4hG) CED 9 
|,yj'©Fe4fl`V€6`l^ivt4{¢^Ty`lx¬es““]_t6eFcTy‘`lx¬€Mjk“pjRhp€6jkh6^ toh‘^i™¢t¦cd]_`l^T`­cRe4f=e4xTx_z~>“^_`ltFhxPÕ +% ¿iÀpÂckesh
rCj,x_zpckc¡jkx_x {Szflflv•e4““]Tt†—`m~¦e4^TjP€™,`m^Ty•eÇaph`l^_j,e4~>tFzh6^Uts{²€e4^ueËFe4hC€BH>¿ +%0IJ ¿ÁÀCÂ=KLK +% ¿ÁÀCÂ_³`lx
x_~¦e4flfjkhtFznoy•x_t'^_yCe4^˜xue4~>“fljkx;{S]TtF~ +% ¿ÁÀC˜e4]_je4flx_tfm`l¨ojkflv^TtrCjŒxde4~•“fmjRx;{S]_to~ +%0IJ ¿ÁÀCÂ
¿S™,yjk]_jGH>¿ KLK M4Â`lx^Tyj’Èzflflrpeoc_¨o¥_q2`ljkrfljk]'€6`l©Fjk]TnFjkhpc¡j‘rpjR^i™¢jkjRh e4hC€NM4Âk(ԟtFzhC€6xtoh
^_yjkx_jcdtFhC€6`l^_`ltFhxŒcRe4h‘rCj zxTjP€‘^Tt“]_t4©ojcdtoh+©ojk]_nojkhpcdj'`lh@eaph`l^Tj h6z~rpjR],t4{=x_zpcRcdjkx_xT`l©Fj
e4““]_t†—`l~’es^_`ltFh'x_^Tjk“xP
|,yj,“Cjk]i{KtF]_~¦e4hpc¡j¢ts{É^Ty`lx es““]Tt+eFcTy'`lx €Mjk“pjRhp€6jkh6^˜tFh'^_yj,hpes^_z]Tj,t4{2^_yjŒ€Mjkhx_`l^ive4““]_t†—6¥
`l~’es^_tF]2zx_jP€2;®!jŸype†©Fj,cTytFxTjkh^Ttjkx_^T`m~¦e4^Tj=^_yj,cdtohp€6`l^_`ltFhCe4f€6`lx_^T]_`lrz^T`mtohx2{KtF];jk©FjR]_v “pes`l]
t4{=“pes]ues~•jk^Tjk]_x,`lh¦^_yj]_jk“]Tjkx_jRh+^de4^T`mtohU2e'^_to^ue4f³t4{PO'¿RQTS†Â˜h6z~rpjR]_xPŸgih‘^_yj¬hjd—^Œx_j†cd^_`ltFh
™˜j™,`lflf;x_yt4™)yt4™™˜jzx_j^_yjkx_j¦cdtFhp€M`m^T`ltFhpesfmx€6`lx_^T]_`lrz^T`mtohx^_t‘cdtohx_^T]_zpcd^'e¬Ö¡tF`lh6^¬€6`lxi¥
^_]T`lrz^_`ltoh@™,y`VcTy8`mx cdfltox_jkxT^`lh8^_yj•Èqªx_jkhx_j•^Tt‘^_yj^_]_zjŒÖdtF`lh6^¬€6`lxT^_]_`lrz^_`ltFhU¤6zCc_y(esh
e4““]_t†—`l~’es^_tF]C`lxXckes“pe4rfmj˜t4{C]_jk“]Tjkx_jRh+^T`lhncdflzxT^_jk]Tx=t4{py`mnoyflvŒ]TjkfVe4^_j†€ “pes]ues~•jk^Tjk]_x†U®Äy`mflj
^_y`mx˜~•`lnoy+^;xTjkjk~ x_`l~•`lfVe4]³^_t'^_yj`lh6^_z`l^_`l©oj,rpjkyCe†©+`lto]=t4{=cd]_tox_x_t4©ojk]P^_y`lx˜]_jk“]_jkxTjkh6^ue4^T`ltFh¦`lx
x_^T]_`Vcd^Tfmvš~•tF]Tj “Ct4™¢jk] {Szfi w tF]_j'`l~•“CtF]T^ue4h6^Tfmv6toz]¬c¡fmzx_^_jR]_xe4]_j ( (Æ{S]_to~±^_yj•€es^ue
e4hC€’es]_jhto^,“]_jd¥_€6jdaChjP€¦r6v•^Tyj “]TtFno]ue4~>~•jk]†
³¸+ ŽU¼ ¸
³¸ ‹kŽ¹ # ¹ ¸ º¼i¼_¹ ¸  Ž¹=‰Œ;‰¼ ¼ ¼ ‹
|,yj=Ödto`mh6^˜“]_torpe4r`mfl`l^iv€6`lx_^_]T`lrz^_`ltoh’t4©ojk]Œex_jk^t4{X]de4hp€MtF~ ©Fe4]_`VesrfljkxP 1= C+6`lxPÕ
C ¿ S K C ¿ I 9 K p ¿ p ¿u„†Â
¿ ‘ ¿ 9K S ½`l©FjRhtohflv¬“Ce4`l]_™,`lx_j,cdtohp€6`l^_`ltFhCe4f6“]_torpe4r`mfl`l^_`ljkx† ¿ C K !FÂ;e4hC€ zhCcdtFhp€M`m^T`ltFhpesf+“]_tFrCe4r`lfœ¥
`l^_`ljkx† ¿ CiÂk6™˜j¬e4]Tj{ÁeFcdj†€™,`l^_y>^_yj^uesx_¨ts{XnFjRhjk]ues^_`lhn'xues~•“fljkx˜^_ypes^¢~¦e4^dc_y•esx§c¡fmtox_jkflv
e4x;“CtFxTx_`lrflj¢^Tyj,^_]Tzj2ÖdtF`lh6^X€6`lxT^_]_`lrz^_`ltFhU ¿ ‘ÂR;gi^X`lx³htF^;“CtFx_xT`lrflj¢^TtÆcRe4“^Tz]_jŒesfmf6“CtFxTx_`lrflj
Ödto`mh6^Œ€6`lxT^_]_`lrz^_`ltFhxts{.Q8©Fe4]T`­esrfljkxŒzx_`lhn•tohflv•^Tyjzhpcdtohp€6`l^_`ltohpe4f;e4hp€¦“Ce4`l]_™,`lx_j'cdtFhp€M`b¥
^_`ltohpe4f6“]TtFrpesr`lfm`l^T`mjRxPË4yt4™˜jk©ojk]Po™¢j,™˜tFzf­€Æfm`l¨oj¢^Tt€6jkxdcd]_`lrCjŒ^_yj,^_]_zjUÖdtF`lh6^=€6`lx_^_]T`lrz^_`ltoh esx
cdfltFxTjkflv’esx,“ptox_x_`lrfljF=ԟjkflt4™ 6™˜j€6jk]T`m©oj e4h‘e4flnFto]_`l^_y~{KtF],c_yt+tox_`lhnxTzpcTy e€Mjkxucd]T`l“^_`ltFh³
½`l©FjRh@e'“pjR]_~z^de4^_`ltoh’ts{¢^_yjh6z~rCjk]_xrpjR^i™¢jkjRhф'e4hp€GQ;# " $ %9 $ S $&22™˜j€6j¡aphje
cdfVe4xTx,t4{=“]TtFrpesr`lfm`l^iv€6`lx_^T]_`lrz^T`mtohxP(*' ) ¿ ‘ÂRÕ
¿ÁƒoÂ
' ) ¿ ‘ÂP ¿ C,+ K C&-k ¿ C&- K C&.k ¿ C0/213+3K C,/F ¿ C,/+ |,yjŒ€6`lxT^_]_`lrz^_`ltFh45' ) ¿ ‘Â=zxTjkx("•e4x˜e4htF]d€6jk]T`mhn{StF];^TyjŒ“pes`l]_™,`lx_j,cdtFhC€6`l^_`ltFhpesf“]TtFrpesr`lfm`œ¥
^_`ljkx†X´z];not+e4fo`lxU^_tŒcTyt6tox_j˜^_yj˜“pjR]_~z^de4^_`ltoh6"'^Type4^³~’e —M`l~`l”kjkxC^_yj§esnF]_jRjk~•jkh6^³rpjk^i™˜jkjkh
5' ) ¿ Â'e4hC€‘^Tyj•^T]_zj•€6`lxT^_]_`lrz^_`ltFh ¿ ÂR•|,yj’e4no]_jkjR~•jkh6^rpjR^i™¢jkjRh(^i™˜t €M`mxT^_]T`mrz^_`ltFhx
ckesh’rCj ~>jPe4xTz]_jP€År6v•^_yjÈzflfmrCeFcT¨o¥ q²`mjRrfljk]§€M`m©ojk]_nojkhpcdjoÕ
H>¿ LK K5 ' ) Â 798 ;: fltFn :!fmton<5' )>=0? @A : flton B= :C@A : flton<*' )D=
:FE²¿  :C@ A : flton ¿ C,+K CG-d ¿ &C -"K C&.k ¿ C,/1H+"K C0/F ¿ C,/F =
F: E²¿ Â;IJE2¿ C,+3K C&-kÂ#IKE²¿ C&-"K &C .kÂ#I ILE²¿ C,/1H+K C0/FÂMIKE²¿ 0C /6 |,y`lx €6`l©Fjk]TnFjkhCcdj`lx e4fl™,e†v6xŒhtoh6¥Áhjkn+es^_`l©FjFC™,`l^_yšjPµozpe4fl`l^iv¦tFhflv¦`lh‘^_yjckesx_j™,yjk]T4
j ' ¿N"2Â
e4hC€ ¿ ‘•es]_j‘`V€6jRh+^T`Vcke4f€6`lx_^_]T`lrz^_`ltohxP|,yj‘to“^_`l~¦e4Of "`lx•€6jdaChjP€e4x^_yj‘tFhjš^_ypes^
~•`lh`l~`l”kjkx˜^_y`mx€6`l©Fjk]TnFjkhCcdjFžto]Œe€6`lx_^_]T`lrz^_`ltoh‘^_ypes^¬ckesh‘rCjcdtF~>“fljk^_jRfmv¦€6jkxdcd]_`lrCjP€‘r6v
“pes`m]T™,`lx_j•cdtohp€6`l^_`ltFhCe4f;“]_torpe4r`mfl`l^_`ljkx†2^_yj•tF“^_`l~’esPf "ª™,`lfmf;nojkhjk]de4^_j¦e‘€6`lx_^T]_`lrz^T`ltFh8^_ypes^
™,`lflfprCj`­€Mjkh6^_`Vcke4fC^_t'^_yj¬^T]_zj €6`lxT^_]_`lrz^_`ltFhU³gihxTt4{Áes]§esx¢^Tyj^_]_zjƀM`mxT^_]T`mrz^_`ltFh¦cke4hhtF^˜rpj
ckes“^_z]TjP€¦^Ty`lx,™§e†v6F^Tyj to“^_`l~’esQf ' ) ¿ ÂǙ,`lfmfU€M`m©ojk]_noj{S]_to~ ^_ypes^Œ€6`lx_^T]_`lrz^T`mtohU
|,yj˜ap]Tx_^=^Tjk]_~)`lh^TyjŒ€6`l©Fjk]TnFjkhCcdjŒ€6t6jkx=hto^¢€6jk“Cjkhp€'to<
h ";³|,yjk]Tjd{Sto]_jF6^TyjŒcdtFxT^;{SzhCcd^_`ltFh³
R ) ¿ ‘Âk™˜j™,`lx_y¦^_t'~•`lh`l~•`œ”kj,`lxPÕ
R ) ¿ ‘ÂPE2¿ C +3K C -kÂ#ILE2¿ C -3K C .kÂ#I IKE²¿ C /1H+K C /FÂMIKE²¿ C /6 ¿Á…oÂ
|,yjto“^_`l~’esfPª
" `lx ^_yjtFhj^Type4^ “]_t€6zpcdjRx^Tyj•flt4™˜jkx_^“pes`m]T™,`lx_jjkh6^_]TtF“6v8™,`m^Ty!]_jkxT“pj†cd^
^_t8^_yj•^_]Tzj‘€6`lx_^T]_`lrz^T`mtohU!Ô=v!xTjPe4]dcTy`lhn!t4©Fjk]e4flf QS;“Cjk]T~z^ues^_`ltFhx†U`l^'`lx“CtFxTx_`lrflj•^Tt
€6jk^Tjk]_~>`lhj^_yj to“^_`l~¦e4f#;" =gih¦^_yj`mh6^Tjk]_jkxT^_xt4{=cdtF~>“z^de4^_`ltohpe4fCjd°’cd`ljkhpc¡v+™¢j jR~•“flt4ve
x_^T]ues`mnoy6^i{Sto]_™,e4]u€ÅnF]TjkjP€6v‘e4flnotF]_`l^Ty~^_t“`VcT¨’e“pjR]_~z^de4^_`ltohUÕ
o„ ($&=~•`lh E²' ¿ !FÂk
ƒ($ =~•`lh E²' ¿ !+K C
+u†6™,yjk]Tj
$ D 9 $ ‘e4hp€ Q :ĄQ :(ƒ dƒ†„F
™,yjk]Tj E²' ¿Á²`lxU^_yj=jk~•“`m]T`Vcke4f4jkh6^T]_tF“6v6X´hpc¡j§e€6`lx_^_]T`lrz^_`ltoh¬`lx;c_ytFx_jRhU4nFjRhjk]ues^_`lhnxue4~>“fljkx
`lxŒe4flx_t'x_^T]ues`mnoy6^i{Sto]_™,e4]u€²Õ
„o,}˜yt6tFx_j'e ©FesfmzjŒ{KtF] C /rpe4xTjP€>tFh¦`l^_x,jk~>“`l]_`Vckesfp“]TtFrpesr`lfl`m^iK
v ' ¿ C /+ ƒŸ{StF
] N Q:„ Q:у dƒ †„F2cTyt6tFxTj•jkfljk~>jkh6^ C rpesx_jP€£tFh8^_yj>jk~•“`l]T`­cRe4f
cdtFhp€M`m^T`ltFhpesf2“]_torpe4r`lfl`l^i4
v ' ¿ C K C +uÂR
|,yjap]Tx_^¬esflnFtF]T`l^_y~]_zhxŒ`lh•^T`l~•
j Å¿RQTS†Â,e4hC€^_yj x_jPc¡tFhp€¦`lh>^_`l~•
j ¿QTSPÂR
Ž¼
2¸ ‹
|;t,~•jPesx_z]Tj=^_yj¢“Cjk]i{KtF]_~¦e4hpc¡j=t4{ w g w g_}†™˜j=“Cjk]i{Sto]_~>jP€^_y]_jkj˜rpjRhpcTy~’es]_¨jd—“pjR]_`l~•jkh6^Tx
e4hC€’c¡tF~•“Ce4]_j†€•toz],]_jRx_zfl^_x™,`l^_y¦^_ytox_j tor^ues`lhjP€>zx_`lhnxTjk©FjR]ue4f;xT^ue4hC€e4]d€¦tF“^T`l~•`l”Pe4^T`ltFh
e4flnotF]_`l^Ty~•x†
®!jŒ™,`lflf2zxTj{StFz]§esfmnotF]T`m^Ty~•xŸ`mh>tFz]Œc¡tF~•“Ce4]_`lxTtFhx†Õ
„o w g w g_} ¥¢^_yj'e4flnFto]_`l^_y~ esrpt4©ojŒ™,`l^_y‘ƒF‚o‚ xde4~>“fljkx¢“Cjk]`m^Tjk]ues^_`ltFh
ƒ =Ô=g_q/¥¢x_^de4hp€e4]u€>“CtF“zf­es^_`ltFh>rpe4xTjP€>`lhpcd]_jR~•jkh6^uesf2fljPe4]Th`lhn
…?FŒÌŒ} ¥¢]de4hp€6to~•`l”kjP€'y`mflfUcdfl`l~r`lhn
,½\"¥ŒexT^ue4hC€e4]d€‘nojkhjk^T`­c'e4flnotF]_`l^Ty~¯™,`l^Ty‘x_`lhnofmj'cd]_tox_x_t4©ojk] e4hp€£„P‚ ~z^ues^_`ltFh]de4^_j
: ! "2ACYX3•W16I#UL
|,yj@žtFz] Uj†e4¨6x¦“]_torfljk~ `lx^de4¨FjRhŠ{S]_to~%¿iÔ¢esflzPÖke!e4hC€Ä},e4]Tzpe4hpe„P‡F‡oÎFÂR ½`l©FjRh esh
$ ¥_€6`l~•jkhx_`ltFhpesf`lh“z^,©ojPcd^TtF]&!% ^_yj{Stoz],“pj†e4¨6x,jk©Fe4flzpes^_`ltFhÅ{Szhpc¡^_`ltFh>`mxŒ€MjdaphjP€še4xPÕ
™,yjk]Tj
'²¿ (% -ŒÂ
Ä~¦e†— : Ò)H$*K¿i‚ ‘% Â+E-, ? ¿d„ ‘%  = I ¿ (% -ŒÂ
¿ Â
Ò)H$.*K¿0/1 @% ÂP h+z~rpjk],ts{X^T]ue4`lfl`lhn/ x,`lh %
¿ÁÎoÂ
E2, ? 3¿ /14‘% ÂP h+z~rpjk],ts{XfljPeo€6`lhn5/ x˜`lh6 %
¿07oÂ
¿ (% -ŒÂ 98 $‚ `btF{³^_yÒ)H$jk]_*K™,¿i‚`lx_ j @% Â;: "e4hC€ E2, ? ¿d„ ’% Â<: ¿ÁÍoÂ
|,yjk]TjŒe4]Tj,^i™¢tnFfltorpe4fo~’e†—`l~’eX{StF];^_y`mx³{Szhpcd^_`ltohUU|,yjkv es]_j,eFcTy`ljk©ojP€'jk`l^_yjR]=™,yjkh^Tyjk]_j
e4]Tj? I8„,fljPeo€6`lhn„ x³{StFflflt4™¢j†€ r6vesfmfC‚ x=tF];™,yjRh•^_yjk]_je4]_j?
I8„,^_]de4`lfl`mhn¬‚ x=“]_jPcdj†€6jP€
r6v/esflf§„ xP>|,yjk]_j¦e4]Tj•e4flx_t‘^i™˜t¦x_zrCtF“^_`l~’esfXfltckesf=~’e —M`l~•e'^Type4^ tckc¡z]™,`l^Ty/e¦xT^_]T`mhn
t4{¢esfmf˜„ xŒto]Œe4flfX‚ xPŒžtF],fVe4]TnFj ©FesflzjkxŒts{P p^Ty`lxŒ“]TtFrfljk~ rCjPcdtF~>jkxŒ`lhpc¡]_jPesx_`lhnFflv¦~•to]_j
€6`œ°’cdzfl^;rCjPcke4zx_jŒ^TyjŒrCe4x_`lh't4{Ue4^T^_]ueocd^_`ltoh'{StF];^_yj,`mhM{Sjk]_`lto]Xfltckesf~’e —M`l~¦e=rCjPcdto~•j˜f­es]_nojk]P
F,jRx_zfl^_xÆ{StF]]_zhh`lhn‘^Tyj•e4flnFto]_`l^_y~•xŒe4]Tjx_yt4™,h!`lhšapnFz]_j‘„FÅgih/esfmfŸ^_]_`Vesfmx†T™§esxx_jk^
^_tšrpj‘„†‚ ±ts{ $ 2^_yj>^_to^ue4fŸh+z~rpjk]ts{§`lh“z^_xPÅ|,yj w g w g_}e4flnotF]_`l^Ty~ cdtFhx_`lx_^Tjkh6^_flv
~’e —`m~`l”kjRx¬^Tyj{Kzhpcd^T`mtoh(™,`l^_y(es““]Tt†—M`l~¦e4^_jRfmv¦tohj•^Tjkh6^_yŠ^_yj>h6z~rpjR] t4{Œjk©Fesflzpe4^T`mtohx
]_j†µFz`m]TjP€¦r6v^_yj x_jPc¡tFhp€¦rCjkxT^¬e4flnotF]_`l^Ty~@
Function Evaluations Required to Maximize 4 Peaks
1200
Thousands of Evaluations
1000
800
MIMIC
PBIL
RHC
GA
600
400
200
0
30
40
50
Inputs
60
70
80
ž³`lnFz]Tj•„FÕ ),z~rpjk],ts{=jk©Fe4flzpes^_`ltFhx,ts{¢^_yjžtoz]i¥ ³jPe4¨‘cdtox_^Ç{Szhpcd^T`ltFh¦{Sto]Œ€6`œ›ÉjR]_jkh6^ e4flnFts¥
]_`l^Ty~•x˜“fltF^T^_jP€Å{Sto]§e'©Fes]_`ljk^ivt4{=“]TtFrfljk~>x=x_`l”kjRxP
: É9 [ (W16I#UL
,| yj ¤6`œ— ³jPes¨6x,“]_torfljk~¯`lx,e'x_fl`lnFy6^˜©Fe4]_`Ves^_`ltFhtoh žtFz] ³jPes¨+x™,yjk]Tj
`œ{ Ò)H$.*K¿Á‚uÀCÂ;: "e4hC€ E2, ? ¿u„ uÀCÂ;: Äto]
)Ò H$.*K¿u„ uÀC; : "e4hC€ E2, ? ¿Á‚uÀCÂ;: ¿ÁÊoÂ
‚
to^_yjR]_™,`lx_j
|,y`lx,{Kzhpcd^T`mtoh¦ype4x^i™¢t•eo€€6`l^_`ltFhCe4f2nFfltorpe4f2~¦e†—`l~’e™,yjk]Tj^Tyjk]_je4]Tj KI„fljPeF€6`lhn•‚ x
{Stofmflt4™˜jP€r6v‘e4flf=„ xtF]™,yjkh‘^Tyjk]_je4]Tj K
I„ ^T]ue4`lfl`lhn¦„ x,“]_j†cdjP€6jP€8r6v‘e4flf;‚ xP˜gih‘^Ty`lx
ckesx_jF³`l^¬`lxhto^¬^Tyj©Fe4flzjRx¬ts{§^Tyj•ckeshp€6`V€e4^Tjkx^_ypes^ `lx`m~>“pto]_^de4h6^Prz^ ^Tyjk`l]x_^_]Tzpcd^Tz]_jFÕ
^_yjap]Tx_^ 4I)„“ptox_`l^_`ltohx x_ytFzfV€8^ues¨Fjtoh!^_yj>xue4~>j©FesflzjF³^_yj>f­esx_^ I"„Å“ptox_`l^_`ltFhx
x_ytFzfV€8^ues¨Fjtoh!^_yj>xue4~>j©FesflzjF³^_yjkxTj•^i™˜t‘nF]TtFz“xxTytFzfV€š^ues¨Fj•toh(€6`œ›ÉjR]_jkh6^'©Fe4flzjkx†
e4hC€•^Tyj~•`V€€6flj,“CtFxT`m^T`ltFhx,xTytFzfV€>^ues¨FjtFh‘e4flf2^_yj¬xde4~>jŒ©Fe4flzjo
F,jRx_zfl^_x{Sto]^_y`lx“]_torfljk~ e4]TjxTyt4™,h8`mhšapnoz]_jƒ\Œx~•`lnFy6^,rCjj¡—M“CjPcd^TjP€2=ԟg_qª“pjR]i¥
{Sto]_~•j†€¦™¢to]_xTj^_ypesh!tFh8^_yj•žtFz]³jPes¨@“]TtFrfljk~ rpj†cke4zxTj•`l^^_jkhC€6x^Tt¦tFxuc¡`mflfVe4^Tj`lh8^_yj
~•`V€€Mfmj˜t4{;^_yjxT“peFcdj™,y`lfljŒcdtoh+^T]ueo€6`Vcd^_to]_v•xT`lnFhpesfmxŸ“zflf2`l^=rpeFcT¨•eshp€'{Sto]_^TyU;|,yj]ueshp€6tF~
cd]TtFx_xTt4©Fjk]tF“Cjk]ues^_`ltFh¦ts{X^_yj½\"tckckesx_`ltFhpesflfmv™,e4xŒesrfljŒ^Tt•cke4“^Tz]_jx_tF~>jŒts{ ^Tyjzhp€6jR]i¥
flv6`mhn‘x_^T]_zpcd^Tz]_jo¢]Tjkx_zfm^T`lhn@`lh(esh(`l~•“]Tt4©FjP€š]_jRf­es^_`l©Fj“Cjk] {StF]T~’e4hCcdj•ts{Œ^_yj‘½\'2\Œx™˜j
jd—“CjPcd^_j†€2^Tyj w g w g_}Še4flnotF]_`l^Ty~"™§esx¢e4rfmj,^Tt cke4“^Tz]_j^_yjzhp€6jR]_flv6`mhn¬xT^_]Tzpcd^_z]_jt4{2^_yj
“]TtFrfljk~8Fe4hp€>cdtF~r`mhj¢`lh6{Sto]_~¦e4^_`ltoh{S]_to~ e4flfp^TyjŒ~¦e†—`l~•eC|,y6zx w g w g_}ŠcdtFhx_`lx_^Tjkh6^_flv
~’e —`m~`l”kjRx¬^Tyj’¤M`b— ³jPe4¨6x{Szhpc¡^_`ltFh!™,`l^_y(es““]Tt†—M`l~¦e4^_jRfmv¦tohja{K^_`ljk^_yª^_yj>h6z~rpjR] t4{
jk©FesfmzCe4^_`ltohx¢]TjPµoz`l]_jP€ r+v^Tyj to^_yjk]ŒesflnFtF]T`l^_y~>xP
¿ % - ŒÂP
$
: I
kGACJSAC34[SB
\ nF]de4“yŠ`lxÈ,¥_}˜tFflto]ue4rfmj>`œ{¬`l^`lx“ptox_x_`lrflj¦^_t!e4xTx_`lnFhŠtohj¦t4{ ŠcdtofltF]_x^Tt!jPeFcTyŠt4{^_yj
±
ht€6jkxt4{=^_yj nF]de4“y¦x_zCc_y8^_yCe4^,ht•eo€kÖkeFc¡jkh6^Œht€6jkxype†©Fj^Tyj xues~•j cdtofmto]PŒjR^_jk]T~•`lh`lhn
™,yjk^Tyjk]‘e8nF]de4“yŠ`lx’ÈÇ¥_}˜tFfltF]de4rflj>`mx>¨6ht4™,hŠ^_t!rCj )+³¥_}˜tF~>“fljk^_jo Ì,jR]_jF™¢j8€6jdaphj
w e — ÈÇ¥ }˜tofltF]_`lhn'^_trCj ^Tyj ^uesx_¨>t4{;aphp€6`lhn•ec¡tFfltF]T`mhn^Type4^~•`lh`l~`l”kjkx=^Tyj h6z~rCjk],t4{
eF€RÖkeFcdjRh+^,“Ce4`l]_xŒcdtofltF]_j†€•^Tyjxues~•jF
F,jRx_zfl^_xÆ{StF]^_y`mx“]_torfljk~
e4]_jxTyt4™,h8`mh8aCnFz]Tj•ƒ®!j'zx_j†€(e>x_zrx_jk^'t4{,nF]de4“yx™,`m^Ty
e>x_`lhnofmjx_tFflz^T`ltFh!¿Kz“@^Tt¦“pjk]T~z^ues^_`ltFhxŒt4{,cdtFfltF]dÂx_t¦^_ypes^^_yjto“^_`l~’esf2x_tofmz^_`ltFh‘`lx €6jd¥
“Cjkhp€6jkh6^ D9 tFh‘^Tyj'x_^_]Tzpcd^Tz]_j>t4{¢^Tyj“Ce4]ues~•jk^Tjk]_x† Ô=j†cke4zxTj•ts{¢^_y`lx† ŸÔ=g_qª“Cjk]i{Sto]_~>x
“Ct+to]_flv6U½\ x³“pjk] {StF]T~"rpjR^_^_jR]=rpj†cke4zxT
j ˜cd]_tox_xTt4©Fjk]=“CtF`lh6^U`lx³]_jk“]Tjkx_jRh+^de4^T`m©oj,t4{pxTtF~>j=t4{
^_yj,zhp€6jk]Tflv+`lhnx_^_]Tzpcd^Tz]_jt4{2^_yj,nF]ues“yx=zx_jP€2˜ž³`mhCe4flflv+ w g w g_}ª“pjR]i{Sto]_~•x;rCjkx_^˜rCjPcke4zx_j
Function Evaluations Required to Maximize 6 Peaks
1300
1000
800
MIMIC
PBIL
RHC
GA
1000
Thousands of Evaluations
Thousands of Evaluations
1200
600
400
200
0
10
20
Function Evaluations Required to Maximize K−Coloring
1200
30
Inputs
40
50
60
800
MIMIC
PBIL
RHC
GA
600
400
200
0
10
15
20
25
Inputs
30
35
40
ž³`lnFz]Tj¢ƒÕ )Çz~rCjk]Uts{MjR©Fe4flzpe4^T`ltFhx2ts{p^_yj,¤6`œ—6¥ ³jPes¨¬cdtox_^³{SzhCcd^_`ltFhÅ¿Sfljd{S^dÂUe4hC€^_yj,ÈÇ¥ }˜tofltF]
cdtox_^,{Kzhpcd^T`mtoh‘¿K]_`lnFy6^uŸ{KtF],e'©Fe4]_`ljk^ivts{=“]_torfljk~ x_`l”kjRxP
`l^,`lx§esrflj¬^Tt•cke4“^_z]Tje4flfCt4{=^_yjxT^_]_zCcd^_z]de4f;]_jRnFzfVe4]T`l^iv™,`m^Ty`lh•^Tyj `lh“z^TxP
# ¹ ¸  »Œ‹6¼_¹ ¸ ‹
®!jype†©oj’€6jkxdcd]_`lrCjP€ w g w g_};e‘ht4©Fjkf˜tF“^T`l~•`l”Pe4^T`ltFh@esfmnotF]T`m^Ty~ ^Type4^cdtoh+©ojk]_nojkx{Áe4xT^_jk]
e4hC€'~•tF]Tj¢]Tjkfl`Ve4rflv'^_ypeshx_jk©ojk]uesf2tF^_yjk]¢j¡—M`lxT^_`lhnesfmnotF]T`m^Ty~•x† w g w g_}ŠeFckc¡tF~•“fm`lxTyjkx=^Ty`lx
`lh^i™˜t ™,e†v6xP;žU`l]Tx_^Po`l^;“pjk] {StF]T~•x=to“^_`l~>`m”†e4^_`ltoh¬r6v xTzpckcdjRx_x_`l©ojkflv’e4““]_t†—`l~’es^_`lhn˜^_yjcdtFh6¥
€6`l^_`ltohpe4f2€6`lx_^T]_`lrz^T`mtoh•ts{U^_yj`lh“z^Tx,nF`l©FjRh’erptozhp€toh•^Tyj cdtFxT^,{SzhCcd^_`ltFh³U|,y]TtFznoytFz^
^_y`mx˜“]TtMc¡jkx_x†p^TyjtF“^T`l~z~"t4{;^_yj cdtox_^Ÿ{Szhpcd^T`ltFh•rCjPcdto~•jkx,no]ueo€6zpe4flflv ~>tF]_jfl`m¨ojkflv+²\Œx,e
]_jRx_zfl^P w g w g_}Š€6`l]_jPcd^Tflv•cdtF~>~zh`mcRe4^_jRxX`lh6{KtF]_~¦e4^T`mtoh e4rCtFz^=^Tyj¬c¡tFx_^Ÿ{SzhCcd^_`ltFh'{K]_tF~)^_yj
jPes]_flv•xT^ue4nojkx,^_t^TyjfVe4^_jR],x_^uesnFjkxt4{;^_yjx_j†e4]ucTyU,¤6j†cdtFhp€² w g w g_}«e4^_^Tjk~•“^_x,^_t•€M`mxdcdt4©FjR]
cdto~•~•toh¦zhp€6jk]Tflv+`lhnšx_^_]Tzpcd^Tz]_j‘e4rCtFz^ tF“^T`l~’e'r6v/c¡tF~•“z^_`lhn>x_jPcdtohp€o¥itF]d€6jk]x_^de4^T`mxT^_`Vcdx
e4hC€•xde4~>“fl`mhnŒ{K]_tF~ e€6`lx_^_]T`lrz^_`ltoh’cdtohx_`lx_^Tjkh6^Œ™,`l^_y¦^TytFxTj¬xT^ues^_`lx_^T`­c¡xP
#2B;A'JK1 C5!16BUD4L
gih¦^_y`lx,]Tjkx_jPes]ucTyU ,ojk]_jR~v j Ô=tohjk^Œ`lx,xTz““CtF]_^TjP€šr+v>^_yj¬´ w zfl^_`V€6`lxuc¡`m“fm`lhpes]_vBF,jd¥
x_j†e4]ucTy ;]TtFnF]de4~t4{U^Tyj >,h`l©ojk]_xT`m^ivBF,jRx_jPes]ucTy¦gih`l^_`Ve4^T`m©ojFF}˜ypes]_fljkx,gixTrpjkflfCr6v•e{Sjkflflt4™,x_y`l“
nF]de4h6^_j†€ r6v\=| |Ñq³e4rx ¥ FÇjkx_jPes]ucTyU6e4hp€;e4zf6Ã`mtofVe˜r+v'´,°’c¡j¢ts{ )Œe†©Fesf"F,jRx_jPes]ucTy•½]uesh6^
),tC)Œ‚o‚F‚„ ¥_‡ 7 ¥¡„¡¥ ‚o…„F„o2½]_jkn½ esfm“Cjk]T`mh¦yjRfm“CjP€>`lh¦^_yj“]Tjk“pe4]de4^T`mtoh¦t4{X^Ty`lx,“pe4“Cjk]†
Ž U¸  ‹
!"##$&%''(")+*,*,-/.0%124365,78:9";1<8,119=7>?A@B!"3,2C9";1<?"96!"D8,119=7>
+8/3,!=79";2E FG1>;H7>6+I!"1KJ3,!"9L6!"H1K871NME10O3,QPL75,1!=7?=79SR,
6H2E#T#
LU#
V3/H1K;W#XY#<Z6!"!"19"9#[#(")6*,*/-,.0I\:;1!"18/1H1K9=7>+8,3/!=79";H2N?'1]>1BV^S<_`Sa,b
a"kIlm a k ` a aoprqHlBsl f a g s,tKu d s` gfhgvi4w m d a`0x66yL1Kz|{V3/!"}6~'?"?"3>0769=73,
@Bc03,d!&defhg3,i,2Nj JH9=7Hd'8€n M<6g >0;d 7Hdg#1K!"c0R,d gAn
L36OW‚Yr(")6*/ƒ,-,.0Y„ e s0prlBsl f a g…fg‡† slˆq`=s,ts ge „`l f ‰c6f s,tŠx j l d o j F ;14M…7>;78‹PL7OŒ
5,1!"?=79SR<G!"1?"?
ŽL7!"}J#9"!=7>}rZ'1069"9'UvEI1>>;7BM#(")6*,‘/’,.0“'J9=72”7•69=73/–R—v72$69"1<~'16+O78H
Š c6fhd6gcd v˜,˜/™$(Bšv-,*,‘/.0›Oœ/ƒ$)œ,‘/™$
žŸ6H8#Ž”(")6*/*,-,.07OOŸ>O72”–7H8A–1+9"?8/1H1K9=7>'?"166!">0;A3,€–331+E>7!">$79 ?"R9";1?=7? JH!"3/–$1K2
3@I}/3,•6 ?I^= wŸ¡Gd t klmA¢ g l d ` g sl f a g s6t n a g k d ` d6gcd a gE£ s c m fgd u d s` gfgi –1?$¤vyYvE‚63/!"6WM/^0(")+*,*/-,.0%107$@¥3,!">12N19V166!"$78”–RJ!"3,–#–7OO79=RA2A9">;7H8^=
X'5,7<FG3/H!"19"•K}R,MMv€Ÿ1!"!"3,1,#MUv1$79"3/!"?#„ e¦ s gc0djIfhg4†[d q`=s,t/¢ g ka`0osl f a g
_`Sa c0djjfhgvi /5,36H2N14‘XL1K5,1!)6*,*/-$ME^=F§¨!"1?"? 2–H!=78,1,